For me it’s most commonly taking an A/B test and plugging the result into an online widget to see if it’s reached statistical significance.

But things like

• Regression Analysis
• Cluster Analysis
• Conjoint Analysis
• Time Series Analysis
• Factor Analysis
• Market Basket Analysis

Do any of you use these?

For those who aren’t familiar,

Here’s some examples:

1. A/B Testing (Split Testing)

   • Example: Testing two different email subject lines to determine which one has a higher open rate.

2. Regression Analysis

   • Example: Analyzing how different factors like price, advertising spend, and seasonality affect sales volume.

3. Cluster Analysis

   • Example: Segmenting customers based on purchasing behavior and demographics to tailor marketing efforts to specific segments.

4. Conjoint Analysis

   • Example: Determining which features of a new product are most important to potential customers.

5. Time Series Analysis

   • Example: Forecasting future sales based on historical sales data.

6. Factor Analysis

   • Example: Identifying underlying factors that influence consumer satisfaction with a product.

7. Market Basket Analysis

   • Example: Discovering that customers who buy bread are also likely to buy butter.

A/B testing is often considered the most universally effective approach. This is because it directly tests the impact of changes on performance metrics. It’s best when used iteratively.

My problem with A/B testing

1. is it doesn’t necessarily give meaningful long term insights.

2. When on thing works, often you don’t know why.

3. A success in an A/B test often not transferable.

E.g. we changed the CTA button shape on one page and it increased conv. rate 5%. So we then changed the shape of it on a bunch of other pages and the conv. rate dropped or stayed flat. It only ever worked on that one page.

Curious if people have seen any real success using these other methods to inform their marketing