A model-free method using statistical distance metrics like Pearson chi-squared and KL divergence to identify important features in highly imbalanced datasets.

Learn how to improve experiment efficiency and metric sensitivity through stratified sampling in data analysis.

Discover the truth about using t-tests in AB testing for abnormal distributions in the IT industry.

The article describes using both direct and indirect feedback methods throughout the product development lifecycle.

A statistical approach to determining whether A/B results are significant or random noise.

Learn how to approach data-driven measurement properly. See what unexpected results we got in a bank and get insights for your own data analytics journey.

A deep dive into user reorders, hidden behavioral patterns, and how aggregated funnels improve A/B test accuracy in non-linear user journeys

Learn how to measure marketing impact without A/B tests using causal inference, Diff-in-Diff, synthetic control, and GeoLift.

In this article, we will explore the intricacies of AB testing on small sample sizes, which can be valuable in B2B settings or products with a limited user base

A practical guide to Propensity Score Matching — learn how to estimate treatment effects without running a traditional A/B test.

Identify common A/B testing pitfalls and learn to over come them.

Causal Impact analysis is a valuable tool, but it comes with its set of limitations that practitioners need to be mindful of.

Why data alone misleads—and how emotion, feedback, and AI create better brand decisions.

As a logical person at the casino. you want to put your money on the machine with the maximum expected return. This is the origin of the multi-armed bandit problem. We will cover the two most basic concept here: Beta distribution and Thompson sampling.

So you’ve got a fantastic idea for improving your product’s interface. Problem is, it’s gonna cost time, money and energy to implement. You’re pretty sure it’s gonna be good, but how can you tell for sure? Simple. Use a “Hallway Usability Test”, which will help you find out early on whether you’re on the right track.

Experimentation designing in the marketplace without AB-Testing using Synthetic Control Groups and Switchbacks.

Explore techniques for accelerating A/B testing, including paired testing, covariance adjustment, stratification, CUPED, CUPAC, and Bayesian approaches.

An overview of AB testing during the design of digital products like UX, digital marketing advertisements and software development.

AB testing is an essential aspect of mobile app development, especially when it comes to subscription-based iOS and Android apps.

The recipe for successful A/B testing is quick computation, no duplication, and no data loss. So, we used Apache Flink and Doris to build our data platform.

Leverage Statsig to build Flutter apps FAST!

Bonferroni correction as a solution for multiple comparisons problem in A/B tests. Here is an explanation of how it works with a simulation written in Python.

Discover how Spotify refines A/B testing with decision rules to improve product experimentation and reduce risks in multi-metric analysis.

Can you use machine learning to improve your UX design? Here are 5 ways to use ML when designing your website.

Talking about the role of user research, examples of when you can skip it and how to know when to use it

Every decision and step we take at a startup is based on some belief. Here's how to validate those beliefs and build products like a scientist ;)

Game publishers may have no idea that they have weak marketability until they soft launch their product, and at this point they have already invested huge resources into the development. Yet, there’s a way to make sure that the game can hit its business goals before writing a single line of code. I’m referring to game concept testing.

Discover how our cashback strategy unexpectedly led to an increase in fraudulent activities. Learn from our A/B test results and insights on preventing fraud.

If you are new to Agility and Retrospective, I’ll offer in this post a novel introduction to it. We will explore how retrospectives and your team organization can take inspiration from A/B Testing.

This guide defines that approach, and how you can use A/B testing for product design improvement.

Ah, so you've decided to use video marketing. A wise choice indeed, seeing as, according to Google, "6 out of 10 people prefer online video platforms to live TV".

The three most common mistakes in A/B testing analysis involve the Mann–Whitney test, bootstrapping, and default Type I and Type II error rates.

Learn why exposure points can make or break your mobile A/B tests, common pitfalls to avoid, and practical tips to improve your app experimentation results.

A veteran growth leader explains when A/B testing drives results—and when it slows your team down. Learn how to balance speed and accuracy.

Spotify standardizes A/B testing with success, guardrail, deterioration, and quality metrics to refine product experimentation and minimize risk.

Have you thought of building an app for your business? If so, this is the smartest business decision you can make. In today’s mobile-first world, you have to target potential customers on mobile. According to Techjury, there are 2.7 billion smartphone users around the world.  The same report says 77% of Americans have smartphones. The time spent per user with digital media on mobile in the US daily in 2017 was 2.3 hours.  This shows latent potential for your business in the digital market.  A report by Statista says the total number of mobile app downloads in 2017 was 197 billion. This highlights the importance of mobile apps as a business tool. If your business doesn’t have an app yet, you risk losing out on this customer-rich market. It is important, however, to note that not every mobile app works.  There are millions of apps on Google’s Android Play Store and the Apple app store.  The competition is stiff, and for your app to stand out, you need the highest converting landing page.

This article explores the Mathematical details of least squares estimator in an unbiased and biased settings due to model specification errors.

In a world of LLM and cutting-edge architectures, linear regression quietly plays a crucial role, and it’s time we shine a light on how it can be beneficial.

How to evaluate product releases without an A/B test. A trustworthy framework using causal inference, Synthetic Control, and rigorous data guardrails.

This article details PCIC’s deployment, A/B test lifts, virtual aisles impact, and future directions for combining category and item insights.

A decision rule framework improves A/B testing by balancing statistical rigor and practicality, ensuring reliable product decisions with controlled error rates.

Combining AB testing and reinforcement learning empowers rapid, data-driven business process changes, addressing failures faster than traditional BPM.

Industry experts say AB-BPM’s DevOps-driven process improvements need human oversight, cultural fit, and platform integration for practical success.

Exploring decision theory, OECs, and clinical trial methods to improve A/B testing. Learn how Spotify standardizes multi-metric experiment analysis.

Industry experts see AB-BPM as promising for structured, rapid process testing—if paired with impact forecasts, human oversight, and strong change management.

Monte Carlo simulations analyze the impact of alpha & power corrections in A/B test decision rules, optimizing error rates for better statistical reliability.

Nyholt’s method improves statistical efficiency by refining error rates and sample size calculations, offering an alternative to Bonferroni-type corrections.

Learn how Spotify’s Decision Rule 2 integrates deterioration and quality metrics to improve A/B test validity and prevent regressions in online experiments.

Learn how UI & IU testing principles, Bonferroni corrections, & power adjustments ensure accurate A/B test decisions with multiple success & guardrail metrics.

A grounded theory and ranking-type Delphi study with BPM experts captured qualitative insights on AB-BPM risks, adoption, and key tool features.

Analyzing false positive rates & impacts of sequential deterioration tests on statistical accuracy using Monte Carlo simulations and Group Sequential Testing.

Improve your online sales with my comprehensive guide on Conversion Rate Optimization in E-commerce. Learn about website design, UX, A/B testing, sales funnels.

Here we have listed Best A/B testing tools to help you improve your digital marketing strategy, find solutions, and engage with your ideal customers.

Explore why "some research" can harm innovation & sound design, plus tools & reads for better UX practices. Empower smarter research decisions today!

Learn how A/B testing decision rules use multiple-testing corrections like Bonferroni to balance Type I and Type II errors in multi-metric experiments.

AB-BPM is promising, but needs human oversight, transparency, and integration for success; expert input reveals new research paths and key tool priorities.

Improving efficiency in hypothesis testing by minimizing overlap in rejection regions for success and guardrail metrics in superiority and inferiority tests.

Boost B2C experiment sensitivity with Cross-Fitted CUPED. Learn how to handle heavy-tailed metrics like ARPU without overfitting. Includes Python code.

Even if you’re still knee-deep in holiday and end-of-year promotions, it makes sense to take time to pause. Now’s the time to reflect on the challenges, opportunities, and accomplishments of 2019—before the crazy starts up again.

Learn why Bayesian A/B testing offers more intuitive insights than traditional stats, & get practical tips and tools for better UX decisions under uncertainty.

Bandit algorithms solve some A/B testing complexity, but hide others. Here's a method to fix that.

The basic idea behind an A/B test is to present a change to a small segment of the overall audience, and see how it impacts their behaviour.

Jooble introduced a no-code approach that made testing faster, more cost-effective, and scalable.

3/31/2025: Top 5 stories on the HackerNoon homepage!

A/B Split Testing takes the guesswork out of optimization.

A/B testing can be one of the highest-ROI tools in growth. It's a major unlock in optimizing a business. I have personally launched hundreds of tests. When I...