Mystique CLI is an Atlassian developed CLI for Optimizely Web. It is a multi-phase project that is currently focusing on improving the development cycle for growth engineers. Currently, Mystique is the standard for developing web experiments at Atlassian, and is capable of a wide variety of operations utilizing Optimizely's REST API. This includes creating, updating, testing, and duplicating experiments/personalization campaigns, as well as "promoting" these entities between Optimizely projects for different environments (e.g. from QA => Prod). It has significantly reduced manual overhead and decreased development time by up to 95% for particular actions.

1. NATHAN CASPAR | SR. GROWTH ENGINEER | @NATHAN CASPAR
Mystique CLI
Minimizing the experiment development cycle

2. Content
hosting
Page
targeting
Audience
definitions
Experiment
bucketing

3. Content
hosting
Page
targeting
Audience
definitions
Experiment
bucketing

4. Content
hosting
Page
targeting
Audience
definitions
Experiment
bucketing

5. Content
hosting
Page
targeting
Audience
definitions
Experiment
bucketing

6. Why take the time and effort
to build out a command line
interface?

8. MANUALLY CREATE EXPERIMENT
MANUALLY UPDATE EXPERIMENT COD
MANUALLY DEPLOY EXPERIMENT
MANUALLY TEST EXPERIMENT

9. IDEALISTIC
WORKFLOW
REALISTIC WORKFLOW

10. MANUALLY CREATE EXPERIMENT
MANUALLY UPDATE EXPERIMENT COD
MANUALLY DEPLOY EXPERIMENT
MANUALLY TEST EXPERIMENT
• Create & add audience targeting
• Create & add page targeting
• Create & add events
• Create variations
• Set traffic allocations
• Develop experiment code in local
• For each variation
• For each page
• Copy local code into web editor
• Save experiment
• Publish experiment
• For each targeted page
• Test page conditions
• Test audience conditions
• Test experiment visuals
• Test for bugs

11. MANUALLY CREATE EXPERIMENT
MANUALLY UPDATE EXPERIMENT COD
MANUALLY DEPLOY EXPERIMENT
MANUALLY TEST EXPERIMENT
• Create & add audience targeting
• Create & add page targeting
• Create & add events
• Create variations
• Set traffic allocations
• Develop experiment code in local
• For each variation
• For each page
• Copy local code into web editor
• Save experiment
• Publish experiment
• For each targeted page
• Test page conditions
• Test audience conditions
• Test experiment visuals
• Test for bugs

12. ANY CHANGE MEANS
A LOT OF SMALL MANUAL
STEPS

13. WHICH ADD UP

14. AND UP…

15. AND UP!

16. MANUAL STEPS ALSO
MEAN DEFECT
OPPORTUNITIES

17. WHAT WE WANT
Consistency
Efficiency
Simplicity

19. Why not use an existing
CLI?

20. W
Design
Customization in how we design experiments
Development
Customization in how we build experiments
Analytics
Customization in how we perform post-analysis

21. Phenomenal
REST API
Extremely
Helpful support team

22. Be the change you seek

23. The many faces of Optimizely

24. Operations
Optimizely UI performance
Optimizely snippet performance
Optimizely maintenance
Developer
Simple development cycle
Keeping repo and Optimizely in sync
Quickly deploying and testing updates
Understanding when/where experiments ran
Accessing Optimizely results and raw data
Communicating all things experimentation
PM / Analyst

26. Developer
Simple development cycle
Keeping repo and Optimizely in sync
Quickly deploying and testing updates
PM / Analyst
Understanding when/where experiments ran
Accessing Optimizely results and raw data
Communicating all things experimentation
Operations
Optimizely UI performance
Optimizely snippet performance
Optimizely maintenance

28. Developer
Simple development cycle
Keeping repo and Optimizely in sync
Quickly deploying and testing updates

29. REPETITIVE
MANUAL STEPS
QUICK SIMPLE
COMMANDS

30. BASIC ACTIONS

31. SETUP
Projects
$ optx setup -pr
TokensTokens
GET /projects

32. {
"Project": {
"key_metrics": [ 12345, 45678, 98765 ],
"prefix": "www.website.com",
"project_id": 3232345,
"qa_audience": 2975832345,
},
"QA Project":
{
"key_metrics": [ 53682, 48154, 70962 ],
"prefix": "qa.stg.website.com",
"project_id": 592823671,
"qa_audience": 8275362345,
}…
}
JSON
everywhere
Project config
Experiment
Personalization
Events

33. $ optx create -p=exp.json
CREATE | UPDATE | PRUNE
P13n Campaigns
TokensA/B Experiments
POST /experiments
Events
Pages
Audiences

34. {
"type": "ab_experiment",
"jira_ticket": "BXPSCI-123",
"optx_project": "QA Project",
"experiment_name":"Opticon",
"experiment_description": "Demo for Opticon”,
"variations": ["Opticon demo"],
"page_paths_ids": ["/", "/software"],
}
JSON
everywhere
Project config
Experiment
Personalizatio
n
Events

35. JSON
everywhere
Project config
Experiment
Personalizatio
n
Events
{
"type": "personalization_campaign",
"jira_ticket": "BXPSCI-123",
"optx_project": "QA Project",
"campaign_name": "Opticon campaign",
"page_paths_ids": ["/software","/try"],
"experiences": [{
"exp_name": "Opticon experience 1",
“variations": [ "first variation”,
"second variation”]
},{
"exp_name": "Opticon experience 2",
"variations": [ "first variation v2",
"second variation v2"]}]
}

36. JSON
everywhere
Project config
Experiment
Personalizatio
n
Events
{
"type": "custom_events",
"project": "QA Project",
"events": [
{"key":"api_key_1", "name":"event name 1"},
{"key":"api_key_2", "name":"event name 2"},
{"key":"api_key_3", "name":"event name 3"}
]
}

37. COMPLEX ACTIONS

38. PROMOTION
EXP
launch
PROD
create
build/test
EXP
launch
QA
create
build/test
PR
EXP
launch
PROD
replicates

39. EXP
launch
PROD
DUPLICATION
EXP
launch
PROD
create
We want confirmation
duplicates
Time to product ionize

40. QA

41. Future Vision
Automating clean up
Handling project JS changes
Exposing experiment metadata
Working to open source