Create A/B Tests

Transcript

Hello and welcome to this module on creating A/B Tests in Adobe Target. My name is Drew Burns. I’m a Principal Product Marketing Manager of the Adobe Target solution, the optimization and personalization solution in the Marketing Cloud, and I’m gonna show you how easy and efficient and fast it is to create A/B Tests in Adobe Target using our three step guided visual workflow. And of course, A/B Tests are very valuable in testing one variation against another, content or design, or several other possible variations to see how each affects your conversion goal as well as other key performance indicators and to filter by audiences and see how they performed in it as well. So learning objectives here are, after completing this training, you will be able to create an A/B activity in Adobe Target. You’ll also be able to allocate traffic using either a manual split in our student’s t-test or to automate traffic allocation using a Bayesian method that can increase potential conversion lift as well as potentially reach statistical confidence faster. So with that, let’s dive into the solution and see how you create an A/B test using our guided visual workflow. So when you click into the solution, this is your home screen, if you will, of activities in the activity list. I’ve got Automated Personalization here, monitoring campaign, and many, many A/B tests. So if I come up here, and I wanna create an A/B test or create an activity rather, I have options to select different activity types and A/B test is one of them. They all follow the same three step guided workflow so once I’ve learned how to do an A/B test actually intuitively will have learned how to do all those others activities types and the steps you need to take to execute them. So it asked me which experience composer I wanna use. There’s a visual one that allows me to browse and navigate on my web and mobile sites and select a location and visually make changes or there’s a form based workflow where I can select the location off site such as an email, where I wanna either test or target content or recommendations or other off site locations where I’ve placed my content. But just for the sake of this demo I will select a URL to load in. This is my homepage of Adobe and look the page then becomes accessible, all the containers on the page within our visual experience composer, and when I click it’s identifying the type of content that I’ve selected here and when I click, it gives me the ability to make lots of changes here. I can edit the source of that image, I can even swap the image from my digital asset manager, maybe I wanna see how this looks as a variation there. I can do a lot of other editing and design changes. Like, let’s say, because of this picture, I wanna go ahead and resize this banner here and the content adjust as I move that. Or, speaking of move, if I wanna take an element or navigation and move it to another location on the page, I can do that. Lots of dramatic changes I can make. I can make as many experience as I’d like to test here. I can go ahead and click in here and see the edits that I’m making to the code and edit the code directly or even put in custom code. I can add in multiple pages, so do a multi-page test. I can also see it in mobile viewport, so see it by device if it’s a responsive design sites, lots of a really rich editing and visualization capabilities in this first of the three step guided workflow here, Create. So once I’m satisfied visually, and code wise if need be, with the experiences I’ve created, I can move on to the second stage which is targeting. So, here you’re seeing me build out my linear guided workflow. And it’s very clearly showing me what is being delivered to the audiences and how it’s being delivered here and allowing me to make modifications. Before I do that, I can go ahead and name this A/B test on homepage.

And then I can go in and, so I’m satisfied with that, I can go ahead and change the audience. It defaults to all visitors but when I go in here, hit the stylist, it takes me into the audience list. And this includes audiences that are pre-built in Adobe Target, audiences that I can go ahead and custom create based on all of my data, custom attributes that I’m bringing in, or other audiences that I’ve sourced from other areas of the Marketing Cloud. So we’ll just pick My Custom Analytics Audience. And it’s that same definition of an audience segment from analytics because of that integration between analytics and target. I can go ahead and decrease the sample size within this audience if I wish. And then here comes Traffic Allocation. So this was one of our learning objectives. I can either leave it to a manual split where I’m manually splitting the traffic between the different experiences and then waiting until the student’s t-test reaches statistical confidence to be confident in the repeatability of the results that I see. Or, I can take a first step in automation within testing where I can auto allocate to the best experience. So this again is a Bayesian approach, which over time, traffic will be allocated into the variation that is more significantly performing or outperforming the other variants. So there is a potential for a higher conversion lift in the activity, as well as potentially reaching statistical confidence faster. So we’ll leave it on auto allocate, and then we come to our third setting. So I can put in my objective. This allows me to communicate what my goal is for this test. I can prioritize the test between other activities that might be running on at a similar location and involving a similar audience to keep mutual exclusivity. I can set a duration or just use the default date and time for when to launch it. Or I can use the default which is when activated or deactivated. And then we define our conversion goal and success metrics. What do we want to be tracking as our goal here? And I’ve got a few different options: I can track based on conversion, if I want to do it based on a revenue metric or engagement, time on site, page depth. But, let’s just go ahead and pick Conversion here. And then what do we mean by that? Is it viewing a page, doing a tag or or clicking an element. Let’s just do clicking an element because it will allow me, actually, to come back to my experience here, and, and go ahead and click a link here, if that’s what I want to track, or multiple links. I want to track that. And I can continue to add additional metrics. I can also add metrics and copy this activity to reuse, again and again, if these are report settings that I’d like to use regularly. Or, I can also, if I have the analytics and target server to server integration active, I can just set one default success metric here and then I’m done. And I can select the report sweep from analytics and apply any audience segment from that report sweep, any analytics success metric from that report sweep to my results and drill down with the full power of analytics right within target or conversely do the same thing within analytics. I can also add audience segments here, if I want to specify who those are or just include many from the activity list to look at in reporting. Or, again, if I’m using analytics I don’t need to do that. I can just apply all the analytics audience segments to the reports and then click and we’re done. Look at how easy and fast that was. Again, a three step guided workflow, I build out this overview that very clearly shows me what I’m doing based on my activity, what’s the location, what’s my goal, the priority, what audience am I delivering and how am I delivering those different experiences. It also gives me experience URLs where I can see those experiences if I want to review them or send the links to others for QA. It gives me reports here as well. It’ll let me know if based on that page if there are other activities that potentially would collide with this one and allowing me to modify my priority, which, thankfully, there isn’t here. So there’s no data. And then a Change Log letting me know what changes were made, when and the dates. So that’s it super simple. And again, once you know how to do an A/B test in that three step guided workflow you can do any other activity. It follows that same intuitive create target, and goals and setting. So, as a summary, you should now be able to create an A/B activity in Adobe Target and also allocate traffic using either a manual split or using automated traffic allocation. Thank you so much. -