{"id":4434,"date":"2021-04-11T12:01:16","date_gmt":"2021-04-11T12:01:16","guid":{"rendered":"https:\/\/conversion.com\/?p=4434"},"modified":"2024-08-20T11:26:47","modified_gmt":"2024-08-20T11:26:47","slug":"better-ab-testing-marketing","status":"publish","type":"post","link":"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/","title":{"rendered":"Beyond A vs. B: How to get better results with better experiment design"},"content":{"rendered":"<section class=\"c-post-content\" data-ref=\"case-content\">\n\t<div class=\"post-content\">\n\t\t<div class=\"post-content__container container container--medium\">\n\t\t\t<aside class=\"post-content__sidebar post-content__sidebar--links\">\n\t\t\t\t<div class=\"sticky-menu\" data-ref=\"case-content-menu\">\n\t\t\t\t\t<h3 class=\"sticky-menu__title\">Contents<\/h3>\n\t\t\t\t\t<ul class=\"sticky-menu\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t                            \t\t\t\t\t\t\t<li class=\"sticky-menu__item\">\n\t\t\t\t\t\t\t\t<a class=\"sticky-menu__item-link  active\" href=\"#introduction\" data-ref=\"case-content-menu-item\">\n\t\t\t\t\t\t\t\t\t<span class=\"arrow\"><svg viewBox=\"0 0 16 20\" fill=\"none\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M10.6875 9.34375V7.34375L13.3438 10L10.6875 12.6562V10.6562H2.65625V9.34375H10.6875Z\" fill=\"#F6876F\"\/><\/svg><\/span>\n\t\t\t\t\t\t\t\t\t<span class=\"sticky-menu__item-text\">Introduction<\/span>\n\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t\t\t\t\t                            \t\t\t\t\t\t\t<li class=\"sticky-menu__item\">\n\t\t\t\t\t\t\t\t<a class=\"sticky-menu__item-link \" href=\"#thebasicsdefiningabmvtandfractionalfactorial\" data-ref=\"case-content-menu-item\">\n\t\t\t\t\t\t\t\t\t<span class=\"arrow\"><svg viewBox=\"0 0 16 20\" fill=\"none\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M10.6875 9.34375V7.34375L13.3438 10L10.6875 12.6562V10.6562H2.65625V9.34375H10.6875Z\" fill=\"#F6876F\"\/><\/svg><\/span>\n\t\t\t\t\t\t\t\t\t<span class=\"sticky-menu__item-text\">The basics: Defining A\/B, MVT, and fractional factorial<\/span>\n\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t\t\t\t\t                            \t\t\t\t\t\t\t<li class=\"sticky-menu__item\">\n\t\t\t\t\t\t\t\t<a class=\"sticky-menu__item-link \" href=\"#fullfactorialormultivariatetestmvt\" data-ref=\"case-content-menu-item\">\n\t\t\t\t\t\t\t\t\t<span class=\"arrow\"><svg viewBox=\"0 0 16 20\" fill=\"none\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M10.6875 9.34375V7.34375L13.3438 10L10.6875 12.6562V10.6562H2.65625V9.34375H10.6875Z\" fill=\"#F6876F\"\/><\/svg><\/span>\n\t\t\t\t\t\t\t\t\t<span class=\"sticky-menu__item-text\">Full factorial or multivariate test (MVT)<\/span>\n\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t\t\t\t\t                            \t\t\t\t\t\t\t<li class=\"sticky-menu__item\">\n\t\t\t\t\t\t\t\t<a class=\"sticky-menu__item-link \" href=\"#theexplorationphase\" data-ref=\"case-content-menu-item\">\n\t\t\t\t\t\t\t\t\t<span class=\"arrow\"><svg viewBox=\"0 0 16 20\" fill=\"none\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M10.6875 9.34375V7.34375L13.3438 10L10.6875 12.6562V10.6562H2.65625V9.34375H10.6875Z\" fill=\"#F6876F\"\/><\/svg><\/span>\n\t\t\t\t\t\t\t\t\t<span class=\"sticky-menu__item-text\">The Exploration Phase<\/span>\n\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t\t\t\t\t                            \t\t\t\t\t\t\t<li class=\"sticky-menu__item\">\n\t\t\t\t\t\t\t\t<a class=\"sticky-menu__item-link \" href=\"#fractionalfactorialdesigninactionacasestudy\" data-ref=\"case-content-menu-item\">\n\t\t\t\t\t\t\t\t\t<span class=\"arrow\"><svg viewBox=\"0 0 16 20\" fill=\"none\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M10.6875 9.34375V7.34375L13.3438 10L10.6875 12.6562V10.6562H2.65625V9.34375H10.6875Z\" fill=\"#F6876F\"\/><\/svg><\/span>\n\t\t\t\t\t\t\t\t\t<span class=\"sticky-menu__item-text\">Fractional factorial design in action: A case study<\/span>\n\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t\t\t\t\t                            \t\t\t\t\t\t\t<li class=\"sticky-menu__item\">\n\t\t\t\t\t\t\t\t<a class=\"sticky-menu__item-link \" href=\"#thebesttestingframeworkforyou\" data-ref=\"case-content-menu-item\">\n\t\t\t\t\t\t\t\t\t<span class=\"arrow\"><svg viewBox=\"0 0 16 20\" fill=\"none\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M10.6875 9.34375V7.34375L13.3438 10L10.6875 12.6562V10.6562H2.65625V9.34375H10.6875Z\" fill=\"#F6876F\"\/><\/svg><\/span>\n\t\t\t\t\t\t\t\t\t<span class=\"sticky-menu__item-text\">The best testing framework for\u00a0you<\/span>\n\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t\t\t<\/ul>\n\t\t\t\t<\/div>\n\t\t\t<\/aside>\n\n\t\t\t<div class=\"post-content__content\">\n\t\t\t\t<ul class=\"post-content__sections\">\n\t\t\t\t\t                        \t\t\t\t\t\t<li class=\"post-content__section\" id=\"introduction\" data-ref=\"case-content-item\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<h2 class=\"post-content__section-title\">You\u2019ve been pushing to do more testing at your organization.<\/h2>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"post-content__section-content\"><p>1.\u00a0<a href=\"#bullet-one\">The basics: Defining A\/B, MVT, and fractional factorial<\/a><br \/>\n2.\u00a0<a href=\"#bullet-two\">Fractional factorial design: The middle ground<\/a><br \/>\n3.\u00a0<a href=\"#bullet-three\">The Exploration Phase<\/a><br \/>\n4.\u00a0<a href=\"#bullet-four\">Fractional factorial design in action<\/a><br \/>\n5.\u00a0<a href=\"#bullet-five\">Case Study [VIDEO]<\/a><br \/>\n6.\u00a0<a href=\"#bullet-six\">Case Study\u2014step-by-step<\/a><br \/>\n7.\u00a0<a href=\"#bullet-seven\">Takeaways<\/a><\/p>\n<p>You believe in marketing backed by science and data, and you have worked to get the executive team at your company on board with a tested strategy. Employing a formal plan will allow you to learn more about your customers and\u00a0<b>grow your business<\/b>.<\/p>\n<p>You run A\/B tests, but you aren\u2019t seeing a substantial conversion rate lift and you\u2019re concerned that results aren\u2019t helping to inform business goals for your management team. You could increase the velocity of your testing to get some quick wins, but\u00a0if you want fast wins, you sacrifice insights.<\/p>\n<p>Instead, you need to reexamine how you are structuring your tests. Because, as Alhan Keser\u00a0writes,<\/p>\n<blockquote>\n<p>If your results are disappointing, it may not only be what you are testing \u2013 it is definitely how you are testing. While there are several factors for success, one of the most important to consider is\u00a0<a href=\"http:\/\/en.wikipedia.org\/wiki\/Design_of_experiments\" target=\"blank\" rel=\"noopener\">Design of Experiments (DOE)<\/a>.<\/p>\n<\/blockquote>\n<p>For this post, I teamed up with Director of Optimization Strategy, Nick So, and Optimization Strategist, Michael St Laurent, to take a deeper look at the\u00a0<b>best ways to structure your experiments<\/b>\u00a0for maximum growth and insights.<\/p>\n<\/div>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"post-content__index\">\n\t\t\t\t\t\t\t\t<h3 class=\"post-content__index-title\">Contents<\/h3>\n\t\t\t\t\t\t\t\t<ul class=\"post-content__index-links\" data-ref=\"case-content-menu\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<li class=\"post-content__index-link\">\n\t\t\t\t\t\t\t\t\t\t\t<a href=\"#introduction\" data-ref=\"case-content-menu-item\">\n\t\t\t\t\t\t\t\t\t\t\t\tIntroduction\n\t\t\t\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<li class=\"post-content__index-link\">\n\t\t\t\t\t\t\t\t\t\t\t<a href=\"#thebasicsdefiningabmvtandfractionalfactorial\" data-ref=\"case-content-menu-item\">\n\t\t\t\t\t\t\t\t\t\t\t\tThe basics: Defining A\/B, MVT, and fractional factorial\n\t\t\t\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<li class=\"post-content__index-link\">\n\t\t\t\t\t\t\t\t\t\t\t<a href=\"#fullfactorialormultivariatetestmvt\" data-ref=\"case-content-menu-item\">\n\t\t\t\t\t\t\t\t\t\t\t\tFull factorial or multivariate test (MVT)\n\t\t\t\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<li class=\"post-content__index-link\">\n\t\t\t\t\t\t\t\t\t\t\t<a href=\"#theexplorationphase\" data-ref=\"case-content-menu-item\">\n\t\t\t\t\t\t\t\t\t\t\t\tThe Exploration Phase\n\t\t\t\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<li class=\"post-content__index-link\">\n\t\t\t\t\t\t\t\t\t\t\t<a href=\"#fractionalfactorialdesigninactionacasestudy\" data-ref=\"case-content-menu-item\">\n\t\t\t\t\t\t\t\t\t\t\t\tFractional factorial design in action: A case study\n\t\t\t\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<li class=\"post-content__index-link\">\n\t\t\t\t\t\t\t\t\t\t\t<a href=\"#thebesttestingframeworkforyou\" data-ref=\"case-content-menu-item\">\n\t\t\t\t\t\t\t\t\t\t\t\tThe best testing framework for\u00a0you\n\t\t\t\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/ul>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t                        \t\t\t\t\t\t<li class=\"post-content__section\" id=\"thebasicsdefiningabmvtandfractionalfactorial\" data-ref=\"case-content-item\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<h2 class=\"post-content__section-title\">The basics: Defining A\/B, MVT, and fractional factorial<\/h2>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"post-content__section-content\"><p>Marketers often use the term \u2018A\/B testing\u2019 to refer to marketing experimentation in general. But there are multiple different ways to structure your experiments. A\/B testing is just one of them.<\/p>\n<div id=\"bullet-one\">\n<p>Let\u2019s look at a few: A\/B testing, A\/B\/n testing, full factorial or multivariate (MVT), and fractional factorial design.<\/p>\n<p><strong>A\/B test<\/strong><\/p>\n<p>In an\u00a0<b>A\/B test<\/b>, you are testing your original page \/ experience (A) against a single variation (B) to see which will result in a higher conversion rate. Variation B might feature a multitude of changes (i.e. a \u2018cluster\u2019) of changes, or an isolated change.<\/p>\n<div class=\"main-content__image-component\" style=\"text-align: center;\"><img decoding=\"async\" class=\"entered lazyloaded aligncenter\" src=\"https:\/\/cdn.conversion.com\/wp-content\/uploads\/2017\/03\/AvsB.jpg\" alt=\"\u201cab\" width=\"599\" height=\"424\" data-lazy-src=\"\/wp-content\/uploads\/2017\/03\/AvsB.jpg\" data-ll-status=\"loaded\" \/><br \/>\n<em>When you change multiple elements in a single variation, you might see lift, but what about insights?<\/em><\/div>\n<\/div>\n<p>In an\u00a0<b>A\/B\/n test<\/b>, you are testing more than two variations of a page at once. \u201cN\u201d refers to the number of versions being tested, anywhere from two versions to the \u201cnth\u201d version.<\/p>\n<\/div>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t                        \t\t\t\t\t\t<li class=\"post-content__section\" id=\"fullfactorialormultivariatetestmvt\" data-ref=\"case-content-item\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<h2 class=\"post-content__section-title\">Full factorial or multivariate test (MVT)<\/h2>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"post-content__section-content\"><p>With full factorial or <b>multivariate testing<\/b>, you are testing each, individual change, isolated one against another, by mixing and matching every possible combination available.<\/p>\n<div id=\"bullet-two\">\n<p>Imagine you want to test a homepage re-design with four changes in a single variation:<\/p>\n<ul>\n<li>Change A: New hero banner<\/li>\n<li>Change B: New call-to-action (CTA) copy<\/li>\n<li>Change C: New CTA color<\/li>\n<li>Change D: New value proposition statement<\/li>\n<\/ul>\n<p>Hypothetically, let\u2019s assume that each change has the following impact on your conversion rate:<\/p>\n<ul>\n<li>Change A = +10%<\/li>\n<li>Change B = +5%<\/li>\n<li>Change C = -25%<\/li>\n<li>Change D = +5%<\/li>\n<\/ul>\n<p>If you were to run a classic A\/B test\u2015your current control page (A) versus a combination of all four changes at once (B)\u2015you would get a hypothetical\u00a0<b>decrease of -5%<\/b>\u00a0overall (10% + 5% \u2013 25% +5%). You would assume that your re-design did not work and most likely discard the ideas.<\/p>\n<p>With a multivariate test, however, each of the following would be a variation:<\/p>\n<div class=\"main-content__image-component\" style=\"text-align: center;\"><img decoding=\"async\" class=\"entered lazyloaded aligncenter\" src=\"https:\/\/cdn.conversion.com\/wp-content\/uploads\/2017\/03\/MVT.jpg\" alt=\"mvt widerfunnel\" width=\"594\" height=\"322\" data-lazy-src=\"\/wp-content\/uploads\/2017\/03\/MVT.jpg\" data-ll-status=\"loaded\" \/><br \/>\n<em>When you change multiple elements in a single variation, you might see lift, but what about insights?<\/em><\/div>\n<p>Multivariate testing is great because it shows you the positive or negative impact of every single change, and every single combination of every change, resulting in the most ideal combination (in this theoretical example: A + B + D).<\/p>\n<p>However, this strategy is difficult to execute in the real world. Even if you have a ton of traffic, it would take more time than most marketers have for a test with 15 variations to reach any kind of\u00a0<a href=\"https:\/\/www.optimizely.com\/optimization-glossary\/statistical-significance\/\" target=\"blank\" rel=\"noopener\">statistical significance<\/a>.<\/p>\n<blockquote>\n<p>The more variations you test, the more your traffic will be split while testing, and the longer it will take for your tests to reach statistical significance. Many companies simply can\u2019t follow the principles of MVT because they don\u2019t have enough traffic.<\/p>\n<p>\u2013\u00a0<a href=\"https:\/\/twitter.com\/AlhanKeser?lang=en\" target=\"blank\" rel=\"noopener\">Alhan Keser<\/a>, Product Manager, AI<\/p>\n<\/blockquote>\n<p>Enter\u00a0<b>fractional factorial experiment design<\/b>. Fractional factorial design allows for the speed of pure A\/B testing combined with the insights of multivariate testing.<\/p>\n<p><strong>Fractional factorial design: The middle ground<\/strong><\/p>\n<p>Fractional factorial design is another method of Design of Experiments. Similar to MVT, fractional factorial design allows you to test more than one element change within the same variation.<\/p>\n<p><em>The greatest difference is that fractional factorial design doesn\u2019t force you to test every possible combination of changes.<\/em><\/p>\n<p>Rather than creating a variation for every combination of changed elements (as you would with MVT), you can design your experiment to focus on specific isolations that you hypothesize will have the biggest impact.<\/p>\n<p>With basic fractional factorial experiment design, you could set up the following variations in our hypothetical example:<\/p>\n<p>VarA: Change A = +10%<br \/>\nVarB: Change A + B = +15%<br \/>\nVarC: Change A + B + C = -10%<br \/>\nVarD: Change A + B + C + D = -5%<\/p>\n<div class=\"main-content__image-component\"><img decoding=\"async\" class=\"entered lazyloaded aligncenter\" src=\"https:\/\/cdn.conversion.com\/wp-content\/uploads\/2017\/03\/Factorial.jpg\" alt=\"Fractional factorial design widerfunnel\" width=\"587\" height=\"645\" data-lazy-src=\"\/wp-content\/uploads\/2017\/03\/Factorial.jpg\" data-ll-status=\"loaded\" \/><br \/>\n<em>In this basic example, variation A features a single change; VarB is built on VarA, and VarC is built on VarB.<\/em><\/div>\n<figure class=\"alignfull\">\n<div class=\"container\">\n<div class=\" row\">\n<div class=\"offset-md-2 col-md-8 col-md-8\">\n<p><b>NOTE:<\/b>\u00a0With fractional factorial design, estimating the value (e.g. conversion rate lift) of each change is a bit more complex than shown above. I\u2019ll explain.<\/p>\n<p>Firstly, let\u2019s imagine that our control page has a baseline conversion rate of 10% and that each variation receives 1,000 unique visitors during your test.<\/p>\n<p>When you estimate the value of change A, you are using your control as a baseline.<\/p>\n<div class=\"main-content__image-component\" style=\"text-align: center;\"><img decoding=\"async\" class=\"entered lazyloaded aligncenter\" src=\"https:\/\/cdn.conversion.com\/wp-content\/uploads\/2017\/03\/chart1.jpg\" alt=\"fractional factorial testing widerfunnel\" width=\"60%\" data-lazy-src=\"\/wp-content\/uploads\/2017\/03\/chart1.jpg\" data-ll-status=\"loaded\" \/><em>Variation A versus the control.<\/em><\/div>\n<p>Given the above information, you would estimate that change A is worth a 10% lift by comparing the 11% conversion rate of variation A against the 10% conversion rate of your control.<\/p>\n<p>The estimated conversion rate lift of change A = (11 \/ 10 \u2013 1) = 10%<\/p>\n<p>But, when estimating the value of change B, variation A must become your new baseline.<\/p>\n<div class=\"main-content__image-component\" style=\"text-align: center;\"><img decoding=\"async\" class=\"entered lazyloaded aligncenter\" src=\"https:\/\/cdn.conversion.com\/wp-content\/uploads\/2017\/03\/chart2.jpg\" alt=\"fractional factorial testing widerfunnel\" width=\"60%\" data-lazy-src=\"\/wp-content\/uploads\/2017\/03\/chart2.jpg\" data-ll-status=\"loaded\" \/><em>Variation B versus variation A.<\/em><\/div>\n<p>The estimated conversion rate lift of change B = (11.5 \/ 11 \u2013 1) = 4.5%<\/p>\n<p>As you can see, the \u2018value\u2019 of change B is slightly different from the 5% difference shown above.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/figure>\n<p>When you structure your tests with fractional factorial design, you can work backwards to isolate the effect of each individual change by comparing variations. But, in this scenario, you have four variations instead of 15.<\/p>\n<blockquote>\n<p>We are essentially nesting A\/B tests into larger experiments so that we can still get results quickly without sacrificing insights gained by isolations.<\/p>\n<p>\u2013 Michael St Laurent, Director of Experimentation Strategy&amp; Product, Conversion<\/p>\n<\/blockquote>\n<p>Then, you would simply re-validate the hypothesized positive results (Change A + B + D) in a standard A\/B test against the original control to see if the numbers align with your prediction.<\/p>\n<p>Fractional factorial allows you to get the best potential lift, with five total variations in two tests, rather than 15 variations in a single multivariate test.<\/p>\n<p>But, wait\u2026<\/p>\n<p>It\u2019s not always that simple. How do you hypothesize which elements will have the biggest impact? How do you choose which changes to combine and which to isolate?<\/p>\n<\/div>\n<\/div>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\n\t\t\t\t\t\t                            <div data-target=\"post-newsletter-anchor\" data-device=\"mobile\"><\/div>\n                        \t\t\t\t\t                        \t\t\t\t\t\t<li class=\"post-content__section\" id=\"theexplorationphase\" data-ref=\"case-content-item\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<h2 class=\"post-content__section-title\">The Exploration Phase<\/h2>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"post-content__section-content\"><div id=\"bullet-two\">\n<p>The answer lies in the Explore (or research gathering) phase of your testing process.<\/p>\n<\/div>\n<div id=\"bullet-three\">\n<p>At Conversion, Explore is an expansive thinking zone, where all options are considered. Ideas are informed by your business context, persuasion principles, digital analytics, user research, and your past test insights and archive.<\/p>\n<p>Experience is the other side to this coin. A seasoned optimization strategist can look at the proposed changes and determine which changes to combine (i.e. cluster), and which changes should be isolated due to risk or potential insights to be gained.<\/p>\n<p>At Conversion, we don\u2019t just invest in the rigorous training of our Strategists. We also have a 10-year-deep test archive that our Strategy team continuously draws upon when determining which changes to cluster, and which to isolate.<\/p>\n<\/div>\n<\/div>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t                        \t\t\t\t\t\t<li class=\"post-content__section\" id=\"fractionalfactorialdesigninactionacasestudy\" data-ref=\"case-content-item\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<h2 class=\"post-content__section-title\">Fractional factorial design in action: A case study<\/h2>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"post-content__section-content\"><div id=\"bullet-three\">\n<p>This case follows two experiments we ran on Annie Selke, a retailer of luxury home-ware goods. Our experiment focuses on testing a product category page. (You may have already read about what we did during this test, but now I\u2019m going to get into the details of\u00a0<em>how<\/em> we did it. It\u2019s a fantastic illustration of fractional factorial design in action.<\/p>\n<\/div>\n<div id=\"bullet-six\">\n<p><strong>Experiment 4.7<\/strong><\/p>\n<p>In the first experiment, we tested three variations against the control. As the experiment number suggests, this was not the first test we ran with Annie Selke, in general. But it is the \u2018first\u2019 test in this story.<\/p>\n<p><b>Variation A<\/b>\u00a0featured an isolated change to the \u2018Sort By\u2019 filters below the image, making it a drop down menu.<\/p>\n<div class=\"main-content__image-component\"><img decoding=\"async\" class=\"entered lazyloaded aligncenter\" src=\"https:\/\/cdn.conversion.com\/wp-content\/uploads\/2017\/03\/ANN_LE4_7_VarA-Blog.jpeg\" alt=\"ab testing marketing example\" width=\"599\" height=\"424\" data-lazy-src=\"\/wp-content\/uploads\/2017\/03\/ANN_LE4_7_VarA-Blog.jpeg\" data-ll-status=\"loaded\" \/><\/div>\n<p style=\"text-align: center;\"><em>Replaced original \u2018Sort By\u2019 categories with a more traditional drop-down menu.<\/em><\/p>\n<p><b>Evidence<\/b><\/p>\n<p>This change was informed by qualitative click map data, which showed low interaction with the original filters. Strategists also theorized that, without context, visitors may not even know that these boxes are filters (based on e-commerce best practices). This variation was built on the control.<\/p>\n<p><b>Variation B<\/b>\u00a0was also built on the control, and featured another isolated change to reduce the left navigation.<\/p>\n<div class=\"main-content__image-component\" style=\"text-align: center;\"><img decoding=\"async\" class=\"entered lazyloaded aligncenter\" src=\"https:\/\/cdn.conversion.com\/wp-content\/uploads\/2017\/03\/ANN_LE4_7_V2_VarB-Blog.jpeg\" alt=\"ab testing marketing example\" width=\"599\" height=\"618\" data-lazy-src=\"\/wp-content\/uploads\/2017\/03\/ANN_LE4_7_V2_VarB-Blog.jpeg\" data-ll-status=\"loaded\" \/><br \/>\n<em>Reduced left-hand navigation.<\/em><\/div>\n<p><b>Evidence<\/b><\/p>\n<p>Click map data showed that most visitors were clicking on \u201cSize\u201d and \u201cPalette\u201d, and past testing had revealed that Annie Selke visitors were sensitive to removing distractions. Plus, the persuasion principle, known as the\u00a0<a href=\"https:\/\/www.ted.com\/talks\/barry_schwartz_on_the_paradox_of_choice\" target=\"blank\" rel=\"noopener\">Paradox of Choice<\/a>, theorizes that more choice = more anxiety for visitors.<\/p>\n<p>Unlike variation B,\u00a0<b>variation C<\/b>\u00a0was\u00a0<em>built on variation A<\/em>, and featured a final isolated change: a collapsed left navigation.<\/p>\n<div class=\"main-content__image-component\" style=\"text-align: center;\"><img decoding=\"async\" class=\"entered lazyloaded aligncenter\" src=\"https:\/\/cdn.conversion.com\/wp-content\/uploads\/2017\/03\/ANN_LE4_7_VarC-Blog.jpeg\" alt=\"Collapsed left-hand filter (built on VarA).\" width=\"601\" height=\"423\" data-lazy-src=\"\/wp-content\/uploads\/2017\/03\/ANN_LE4_7_VarC-Blog.jpeg\" data-ll-status=\"loaded\" \/><br \/>\n<em>Collapsed left-hand filter (built on VarA).<\/em><\/div>\n<p><b>Evidence<\/b><\/p>\n<p>This variation was informed by the same evidence as variation B.<\/p>\n<p><b>Results<\/b><\/p>\n<p>Variation A (built on the control) saw a\u00a0<b>decrease in transactions of -23.2%<\/b>.<br \/>\nVariation B (built on the control) saw no change.<br \/>\nVariation C (built on variation A) saw a\u00a0<b>decrease in transactions of -1.9%<\/b>.<\/p>\n<p>But wait! Because variation C was built on variation A, we knew that the estimated value of change C (the collapsed filter), was 19.1%.<\/p>\n<p>The next step was to validate our estimated lift of 19.1% in a follow up experiment.<\/p>\n<h3>Experiment 4.8<\/h3>\n<p>The follow-up test also featured three variations versus the original control. Because, you should never waste the opportunity to gather more insights!<\/p>\n<p><b>Variation A<\/b>\u00a0was our validation variation. It featured the collapsed filter (change C) from 4.7\u2019s variation C, but maintained the original \u2018Sort By\u2019 functionality from 4.7\u2019s control.<\/p>\n<div class=\"main-content__image-component\" style=\"text-align: center;\"><img decoding=\"async\" class=\"entered lazyloaded aligncenter\" src=\"https:\/\/cdn.conversion.com\/wp-content\/uploads\/2017\/03\/ANN_LE4_8_VarA-Blog.jpeg\" alt=\"ab testing marketing example\" width=\"596\" height=\"447\" data-lazy-src=\"\/wp-content\/uploads\/2017\/03\/ANN_LE4_8_VarA-Blog.jpeg\" data-ll-status=\"loaded\" \/><br \/>\n<em>Collapsed filter &amp; original \u2018Sort By\u2019 functionality.<\/em><\/div>\n<p><b>Variation B<\/b>\u00a0was built on variation A, and featured two changes emphasizing visitor fascination with colors. We 1) changed the left nav filter from \u201cpalette\u201d to \u201ccolor\u201d, and 2) added color imagery within the left nav filter.<\/p>\n<div class=\"main-content__image-component\" style=\"text-align: center;\"><img decoding=\"async\" class=\"entered lazyloaded aligncenter\" src=\"https:\/\/cdn.conversion.com\/wp-content\/uploads\/2017\/03\/ANN_LE4_8_VarB_Dropdown-Blog.jpeg\" alt=\"ab testing marketing example\" width=\"597\" height=\"442\" data-lazy-src=\"\/wp-content\/uploads\/2017\/03\/ANN_LE4_8_VarB_Dropdown-Blog.jpeg\" data-ll-status=\"loaded\" \/><br \/>\n<em>Updated \u201cpalette\u201d to \u201ccolor\u201d, and added color imagery. (A variation featuring two clustered changes).<\/em><\/div>\n<p><b>Evidence<\/b><\/p>\n<p>Click map data suggested that Annie Selke visitors are most interested in refining their results by color, and past test results also showed visitor sensitivity to color.<\/p>\n<p><b>Variation C<\/b>\u00a0was built on variation A, and featured a single isolated change: we made the collapsed left nav persistent as the visitor scrolled.<\/p>\n<div class=\"main-content__image-component\" style=\"text-align: center;\"><img decoding=\"async\" class=\"entered lazyloaded aligncenter\" src=\"https:\/\/cdn.conversion.com\/wp-content\/uploads\/2017\/03\/ANN_LE4_8_VarC_Persistent_Selector-Blog.jpeg\" alt=\"ab testing marketing example\" width=\"599\" height=\"464\" data-lazy-src=\"\/wp-content\/uploads\/2017\/03\/ANN_LE4_8_VarC_Persistent_Selector-Blog.jpeg\" data-ll-status=\"loaded\" \/><br \/>\n<em>Made the collapsed filter persistent.<\/em><\/div>\n<p><b>Evidence<\/b><\/p>\n<p>Scroll maps and click maps suggested that visitors want to scroll down the page, and view many products.<\/p>\n<p><b>Results<\/b><\/p>\n<p>Variation A led to a\u00a0<b>15.6% increase<\/b>\u00a0in transactions, which is pretty close to our estimated 19% lift, validating the value of the collapsed left navigation!<\/p>\n<p>Variation B was the big winner, leading to a\u00a0<b>23.6% increase<\/b>\u00a0in transactions. Based on this win, we could estimate the value of the emphasis on color.<\/p>\n<p>Variation C resulted in a 9.8% increase in transactions, but because it was built on variation A (not on the control), we learned that the persistent left navigation was actually responsible for a\u00a0<b>decrease in transactions of -11.2%<\/b>.<\/p>\n<p>This is what fractional factorial design looks like in action: big wins, and big insights, informed by human intelligence.<\/p>\n<\/div>\n<\/div>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t                        \t\t\t\t\t\t<li class=\"post-content__section\" id=\"thebesttestingframeworkforyou\" data-ref=\"case-content-item\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<h2 class=\"post-content__section-title\">The best testing framework for\u00a0you<\/h2>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"post-content__section-content\"><div id=\"bullet-six\">\n<p><strong>What are your testing goals?<\/strong><\/p>\n<\/div>\n<div id=\"bullet-seven\">\n<p>If you are in a situation where potential revenue gains outweigh the potential insights to be gained or your test has little long-term value, you may want to go with a standard A\/B cluster test.<\/p>\n<p>If you have a sufficient amount of traffic, and value insights above everything, multivariate may be for you.<\/p>\n<p>If you want the growth-driving power of pure A\/B testing, as well as insightful takeaways about your customers, you may want to explore fractional factorial design.<\/p>\n<p>Words of encouragement: With fractional factorial design, your tests will get better as you continue to test. With every test you execute, you will learn more about how your customers purchasing behvior\u2014making subsequent experiments more impactful.<\/p>\n<blockquote>\n<p>One 10% win without insights may turn heads your direction now, but a test that delivers insights can turn into five 10% wins down the line. It\u2019s similar to the\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Compound_interest\" target=\"blank\" rel=\"noopener\">compounding effect<\/a>: collecting insights now can mean massive payouts over time.<\/p>\n<p>\u2013 Michael St Laurent<\/p>\n<\/blockquote>\n<\/div>\n<\/div>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t\n\t\t\t\t\t\t                            <div data-target=\"post-newsletter-anchor\" data-device=\"mobile\"><\/div>\n                        \t\t\t\t\t\n\t\t\t\t<\/ul>\n\t\t\t<\/div>\n\n\t\t\t<aside class=\"post-content__sidebar post-content__sidebar--form\">\n\t\t\t\t<div class=\"sticky-newsletter\" data-target=\"post-newsletter-anchor\" data-device=\"desktop\">\n                \t<div class=\"post-newsletter\" data-target=\"post-newsletter\">\n    <h4 class=\"post-newsletter__title\">Real-world growth experiments. In your inbox. Every week. <\/h4>\n            <div class=\"post-newsletter__form post-newsletter__form--world kam-world\">\n        <script charset=\"utf-8\" type=\"text\/javascript\" src=\"\/\/js.hsforms.net\/forms\/embed\/v2.js\"><\/script> <script>   hbspt.forms.create({     region: \"na1\",     portalId: \"9358319\",     formId: \"4e927e03-4f90-466a-8646-7f94947f860c\"   }); <\/script>\n    <\/div>\n            <div class=\"post-newsletter__form post-newsletter__form--uk-us kam-uk-us\">\n            <script charset=\"utf-8\" type=\"text\/javascript\" src=\"\/\/js.hsforms.net\/forms\/embed\/v2.js\"><\/script><script>hbspt.forms.create({ region: \"na1\", portalId: \"9358319\", formId: \"03e799e7-ee63-4857-9182-cb3a687dba40\" });<\/script>\n        <\/div>\n        <div class=\"post-newsletter__terms\"><p>Join 5,000+ optimizers who subscribe to our content<\/p>\n<\/div>\n<\/div>\n\t\t\t\t<\/div>\n            <\/aside>\n\t\t<\/div>\n\t<\/div>\n<\/section>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":60,"featured_media":4439,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[12],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Beyond A vs. B: How to get better results with better experiment design | Conversion<\/title>\n<meta name=\"description\" content=\"If you want the growth-driving power of pure A\/B testing, take a deeper look at the best ways to structure your experiments for maximum growth and insights.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Beyond A vs. B: How to get better results with better experiment design | Conversion\" \/>\n<meta property=\"og:description\" content=\"If you want the growth-driving power of pure A\/B testing, take a deeper look at the best ways to structure your experiments for maximum growth and insights.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/\" \/>\n<meta property=\"og:site_name\" content=\"Conversion\" \/>\n<meta property=\"article:published_time\" content=\"2021-04-11T12:01:16+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-08-20T11:26:47+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/conversion.com\/wp-content\/uploads\/2023\/04\/beyond-a-vs-b_blog_1600x1000_COMPRESSED-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"751\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Nick So\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/conversion.com\/wp-content\/uploads\/2023\/04\/beyond-a-vs-b_blog_1600x1000_COMPRESSED-1.png\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Nick So\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/\"},\"author\":{\"name\":\"Nick So\",\"@id\":\"https:\/\/conversion.com\/#\/schema\/person\/0dbd5e29f8e3e49c6af48f925c571963\"},\"headline\":\"Beyond A vs. B: How to get better results with better experiment design\",\"datePublished\":\"2021-04-11T12:01:16+00:00\",\"dateModified\":\"2024-08-20T11:26:47+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/\"},\"wordCount\":13,\"publisher\":{\"@id\":\"https:\/\/conversion.com\/#organization\"},\"articleSection\":[\"Conversion Optimization\"],\"inLanguage\":\"en-GB\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/\",\"url\":\"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/\",\"name\":\"Beyond A vs. B: How to get better results with better experiment design | Conversion\",\"isPartOf\":{\"@id\":\"https:\/\/conversion.com\/#website\"},\"datePublished\":\"2021-04-11T12:01:16+00:00\",\"dateModified\":\"2024-08-20T11:26:47+00:00\",\"description\":\"If you want the growth-driving power of pure A\/B testing, take a deeper look at the best ways to structure your experiments for maximum growth and insights.\",\"breadcrumb\":{\"@id\":\"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/conversion.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Beyond A vs. B: How to get bet&hellip;\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/conversion.com\/#website\",\"url\":\"https:\/\/conversion.com\/\",\"name\":\"Conversion\",\"description\":\"Better results with evidence\",\"publisher\":{\"@id\":\"https:\/\/conversion.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/conversion.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/conversion.com\/#organization\",\"name\":\"Conversion\",\"url\":\"https:\/\/conversion.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/conversion.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/conversion.com\/wp-content\/uploads\/2022\/12\/Colour-Logo.png\",\"contentUrl\":\"https:\/\/conversion.com\/wp-content\/uploads\/2022\/12\/Colour-Logo.png\",\"width\":1821,\"height\":193,\"caption\":\"Conversion\"},\"image\":{\"@id\":\"https:\/\/conversion.com\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/conversion.com\/#\/schema\/person\/0dbd5e29f8e3e49c6af48f925c571963\",\"name\":\"Nick So\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/conversion.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/7bc1a572c4703989c30e9da5f6d90a78?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/7bc1a572c4703989c30e9da5f6d90a78?s=96&d=mm&r=g\",\"caption\":\"Nick So\"},\"url\":\"https:\/\/conversion.com\/blog\/author\/nick-so\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Beyond A vs. B: How to get better results with better experiment design | Conversion","description":"If you want the growth-driving power of pure A\/B testing, take a deeper look at the best ways to structure your experiments for maximum growth and insights.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/","og_locale":"en_GB","og_type":"article","og_title":"Beyond A vs. B: How to get better results with better experiment design | Conversion","og_description":"If you want the growth-driving power of pure A\/B testing, take a deeper look at the best ways to structure your experiments for maximum growth and insights.","og_url":"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/","og_site_name":"Conversion","article_published_time":"2021-04-11T12:01:16+00:00","article_modified_time":"2024-08-20T11:26:47+00:00","og_image":[{"width":1200,"height":751,"url":"https:\/\/conversion.com\/wp-content\/uploads\/2023\/04\/beyond-a-vs-b_blog_1600x1000_COMPRESSED-1.png","type":"image\/png"}],"author":"Nick So","twitter_card":"summary_large_image","twitter_image":"https:\/\/conversion.com\/wp-content\/uploads\/2023\/04\/beyond-a-vs-b_blog_1600x1000_COMPRESSED-1.png","twitter_misc":{"Written by":"Nick So"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/#article","isPartOf":{"@id":"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/"},"author":{"name":"Nick So","@id":"https:\/\/conversion.com\/#\/schema\/person\/0dbd5e29f8e3e49c6af48f925c571963"},"headline":"Beyond A vs. B: How to get better results with better experiment design","datePublished":"2021-04-11T12:01:16+00:00","dateModified":"2024-08-20T11:26:47+00:00","mainEntityOfPage":{"@id":"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/"},"wordCount":13,"publisher":{"@id":"https:\/\/conversion.com\/#organization"},"articleSection":["Conversion Optimization"],"inLanguage":"en-GB"},{"@type":"WebPage","@id":"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/","url":"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/","name":"Beyond A vs. B: How to get better results with better experiment design | Conversion","isPartOf":{"@id":"https:\/\/conversion.com\/#website"},"datePublished":"2021-04-11T12:01:16+00:00","dateModified":"2024-08-20T11:26:47+00:00","description":"If you want the growth-driving power of pure A\/B testing, take a deeper look at the best ways to structure your experiments for maximum growth and insights.","breadcrumb":{"@id":"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/conversion.com\/blog\/better-ab-testing-marketing\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/conversion.com\/"},{"@type":"ListItem","position":2,"name":"Beyond A vs. B: How to get bet&hellip;"}]},{"@type":"WebSite","@id":"https:\/\/conversion.com\/#website","url":"https:\/\/conversion.com\/","name":"Conversion","description":"Better results with evidence","publisher":{"@id":"https:\/\/conversion.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/conversion.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Organization","@id":"https:\/\/conversion.com\/#organization","name":"Conversion","url":"https:\/\/conversion.com\/","logo":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/conversion.com\/#\/schema\/logo\/image\/","url":"https:\/\/conversion.com\/wp-content\/uploads\/2022\/12\/Colour-Logo.png","contentUrl":"https:\/\/conversion.com\/wp-content\/uploads\/2022\/12\/Colour-Logo.png","width":1821,"height":193,"caption":"Conversion"},"image":{"@id":"https:\/\/conversion.com\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/conversion.com\/#\/schema\/person\/0dbd5e29f8e3e49c6af48f925c571963","name":"Nick So","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/conversion.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/7bc1a572c4703989c30e9da5f6d90a78?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/7bc1a572c4703989c30e9da5f6d90a78?s=96&d=mm&r=g","caption":"Nick So"},"url":"https:\/\/conversion.com\/blog\/author\/nick-so\/"}]}},"_links":{"self":[{"href":"https:\/\/conversion.com\/wp-json\/wp\/v2\/posts\/4434"}],"collection":[{"href":"https:\/\/conversion.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/conversion.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/conversion.com\/wp-json\/wp\/v2\/users\/60"}],"replies":[{"embeddable":true,"href":"https:\/\/conversion.com\/wp-json\/wp\/v2\/comments?post=4434"}],"version-history":[{"count":3,"href":"https:\/\/conversion.com\/wp-json\/wp\/v2\/posts\/4434\/revisions"}],"predecessor-version":[{"id":7339,"href":"https:\/\/conversion.com\/wp-json\/wp\/v2\/posts\/4434\/revisions\/7339"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/conversion.com\/wp-json\/wp\/v2\/media\/4439"}],"wp:attachment":[{"href":"https:\/\/conversion.com\/wp-json\/wp\/v2\/media?parent=4434"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/conversion.com\/wp-json\/wp\/v2\/categories?post=4434"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}