Interrupted Time Series for fixed-period interventions#
Suppose you run a 12-week marketing campaign and sales jump while the campaign is active. The next question is usually harder: does the effect disappear immediately, or does some of it persist after the campaign ends?
This notebook shows how to answer that fixed-period version of interrupted time series (ITS) in CausalPy. We will:
inspect the simulated data and the raw time series pattern
estimate the during-campaign and post-campaign effects separately
summarize persistence and decay
run sensitivity checks to see whether the estimated effect looks distinctive
Providing treatment_end_time tells CausalPy that the intervention has a fixed duration. That creates three distinct periods:
Pre-intervention period: Before any treatment has taken effect. This is the period used to learn the baseline trend and seasonality.
Intervention period: When treatment is active (from
treatment_timetotreatment_end_time)Post-intervention period: After treatment ends
This notebook is a worked example of the three-period case. If you are brand new to interrupted time series in CausalPy, start with Bayesian Interrupted Time Series for the standard two-period setup, then come back here for fixed-duration interventions.
Note
The main new idea in this notebook is not just detecting whether something changed at treatment start, but separating the during-treatment effect from the post-treatment carryover.
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import causalpy as cp
%load_ext autoreload
%autoreload 2
%config InlineBackend.figure_format = 'retina'
seed = 42
Example: Marketing Campaign#
We simulate a 12-week marketing campaign with an immediate effect of about +25 units while the campaign is running and a smaller post-campaign lift of about +8 units afterward.
Because this is simulated data, we know the ground truth. That makes this notebook easier to learn from: we can compare the model summaries and sensitivity checks against a story where we already know there is a real effect, and we know that the post-intervention effect should be smaller than the during-intervention effect.
Show code cell source
# Set up simulation parameters
rng = np.random.default_rng(seed)
n_weeks = 135 # 2 years of weekly data
dates = pd.date_range(start="2022-06-01", end="2024-12-31", freq="W")
# Baseline: trend + seasonality + noise
trend = np.linspace(100, 120, n_weeks)
season = 10 * np.sin(2 * np.pi * np.arange(n_weeks) / 52) # Annual seasonality
noise = rng.normal(0, 5, n_weeks)
baseline = trend + season + noise
# Add intervention effect
treatment_idx = n_weeks // 2 # Start at midpoint
treatment_end_idx = treatment_idx + 12 # 12 weeks duration
y = baseline.copy()
y[treatment_idx:treatment_end_idx] += 25 # During intervention
y[treatment_end_idx:] += 8 # Post-intervention (persistence)
# Create DataFrame
df = pd.DataFrame(
{
"y": y,
"t": np.arange(n_weeks),
"month": dates.month,
},
index=dates,
)
treatment_time = dates[treatment_idx]
treatment_end_time = dates[treatment_end_idx]
print(f"Treatment starts: {treatment_time}")
print(f"Treatment ends: {treatment_end_time}")
print(f"Intervention period: {treatment_end_idx - treatment_idx} weeks")
print(f"Post-intervention period: {n_weeks - treatment_end_idx} weeks")
Treatment starts: 2023-09-17 00:00:00
Treatment ends: 2023-12-10 00:00:00
Intervention period: 12 weeks
Post-intervention period: 56 weeks
Before fitting any model, look for three qualitative patterns in the raw series:
the baseline trend and seasonality before treatment
a jump when the campaign starts
whether the series settles back to baseline or stays somewhat elevated after the campaign ends
The raw plot cannot establish causality on its own, but it gives us a visual target for the model-based summaries that follow.
Show code cell source
# Plot the raw data with treatment periods marked
fig, ax = plt.subplots(figsize=(10, 4))
ax.plot(df.index, df["y"], "o-", markersize=3, alpha=0.6, label="Observed")
ax.axvline(
treatment_time, color="red", linestyle="-", linewidth=2, label="Treatment starts"
)
ax.axvline(
treatment_end_time,
color="orange",
linestyle="--",
linewidth=2,
label="Treatment ends",
)
ax.set_xlabel("Date")
ax.set_ylabel("y")
ax.set_title("Time series observations with a fixed-period intervention")
ax.legend()
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
Run the Analysis#
The model learns the pre-intervention trend and seasonality, then asks what would likely have happened during and after the campaign if that pre-period pattern had continued with no intervention.
Supplying treatment_end_time tells CausalPy to report separate effects for the active intervention period and the post-intervention period:
result = cp.InterruptedTimeSeries(
df,
treatment_time=treatment_time,
treatment_end_time=treatment_end_time,
formula="y ~ 1 + t + C(month, levels=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])",
model=cp.pymc_models.LinearRegression(
sample_kwargs={
"random_seed": seed,
"progressbar": False,
}
),
)
Initializing NUTS using jitter+adapt_diag...
Multiprocess sampling (4 chains in 4 jobs)
NUTS: [beta, y_hat_sigma]
Sampling 4 chains for 1_000 tune and 1_000 draw iterations (4_000 + 4_000 draws total) took 2 seconds.
Sampling: [beta, y_hat, y_hat_sigma]
Sampling: [y_hat]
Sampling: [y_hat]
Sampling: [y_hat]
Sampling: [y_hat]
Visualization#
The three-period design visualization adds a vertical line to mark where the treatment ends:
Solid red line:
treatment_time(intervention start)Dashed orange line:
treatment_end_time(intervention end)
The plot shows three panels:
Top panel: Time series with observations, counterfactual predictions, and causal impact shading
Middle panel: Pointwise causal impact over time
Bottom panel: Cumulative causal impact
The vertical line at treatment_end_time clearly separates the intervention period from the post-intervention period, allowing you to visually assess effect persistence and decay.
fig, ax = result.plot()
plt.show()
Period-Specific Summaries#
The next two outputs answer the core causal question for each period: how different was the observed series from the model’s no-treatment counterfactual?
As you read them, focus on three pieces first:
the estimated effect size
the uncertainty interval around that effect
whether the post-intervention effect is clearly smaller than the during-intervention effect
effect_summary(...) also reports posterior probabilities and relative percent changes, but the effect size and interval usually provide the clearest first read:
# Intervention period
intervention_summary = result.effect_summary(period="intervention")
print(intervention_summary.text)
During the During intervention (2023-09-17 00:00:00 to 2023-12-03 00:00:00), the response variable had an average value of approx. 140.87. By contrast, in the absence of an intervention, we would have expected an average response of 116.34. The 95% interval of this counterfactual prediction is [112.79, 119.68]. Subtracting this prediction from the observed response yields an estimate of the causal effect the intervention had on the response variable. This effect is 24.53 with a 95% interval of [21.18, 28.08].
Summing up the individual data points during the During intervention, the response variable had an overall value of 1690.40. By contrast, had the intervention not taken place, we would have expected a sum of 1396.02. The 95% interval of this prediction is [1353.49, 1436.21].
The 95% HDI of the effect [21.18, 28.08] does not include zero. The posterior probability of an increase is 1.000. Relative to the counterfactual, the effect represents a 21.11% change (95% HDI [17.53%, 24.71%]).
This analysis assumes that the relationship between the time-based predictors and the response observed during the pre-intervention period remains stable throughout the post-intervention period. If the formula includes external covariates, it further assumes they were not themselves affected by the intervention. We recommend inspecting model fit, examining pre-intervention trends, and conducting sensitivity analyses (e.g., placebo tests) to support any causal conclusions drawn from this analysis.
# Post-intervention period
post_summary = result.effect_summary(period="post")
print(post_summary.text)
During the Post-intervention (2023-12-10 00:00:00 to 2024-12-29 00:00:00), the response variable had an average value of approx. 122.63. By contrast, in the absence of an intervention, we would have expected an average response of 116.27. The 95% interval of this counterfactual prediction is [111.85, 120.18]. Subtracting this prediction from the observed response yields an estimate of the causal effect the intervention had on the response variable. This effect is 6.36 with a 95% interval of [2.45, 10.78].
Summing up the individual data points during the Post-intervention, the response variable had an overall value of 6867.34. By contrast, had the intervention not taken place, we would have expected a sum of 6510.97. The 95% interval of this prediction is [6263.45, 6729.98].
The 95% HDI of the effect [2.45, 10.78] does not include zero. The posterior probability of an increase is 0.998. Relative to the counterfactual, the effect represents a 5.51% change (95% HDI [1.72%, 9.28%]).
This analysis assumes that the relationship between the time-based predictors and the response observed during the pre-intervention period remains stable throughout the post-intervention period. If the formula includes external covariates, it further assumes they were not themselves affected by the intervention. We recommend inspecting model fit, examining pre-intervention trends, and conducting sensitivity analyses (e.g., placebo tests) to support any causal conclusions drawn from this analysis.
Comparison Summary#
Once we have separate during-intervention and post-intervention estimates, we can ask a more focused persistence question: how large is the post-intervention effect relative to the during-intervention effect?
period="comparison" gives the quickest plain-language answer to that question:
comparison_summary = result.effect_summary(period="comparison")
print(comparison_summary.text)
Effect persistence: The post-intervention effect (6.4, 95% HDI [2.5, 10.8]) was 25.9% of the intervention effect (24.5, 95% HDI [21.2, 28.1]), with a posterior probability of 1.00 that some effect persisted beyond the intervention period.
The comparison summary provides the fastest persistence read:
Post-intervention effect as a percentage of the intervention effect
Posterior probability that some effect persisted
HDI interval comparison between periods
Use it when you want the headline persistence story in one sentence. The next section shows how to access the same ideas more explicitly in code.
Detailed Persistence Analysis#
If you want to report or reuse persistence quantities in code, analyze_persistence() returns the same core story in a more structured form.
Treat this as a programmatic version of the comparison summary, not a separate inferential step:
persistence = result.analyze_persistence()
============================================================
Effect Persistence Analysis
============================================================
During intervention period:
Mean effect: 24.53
95% HDI: [21.18, 28.08]
Total effect: 294.37
Post-intervention period:
Mean effect: 6.36
95% HDI: [2.45, 10.78]
Total effect: 356.37
Persistence ratio: 0.259
(25.9% of intervention effect persisted)
============================================================
# Access the returned dictionary:
print("\nAccessing results programmatically:")
print(f" Mean effect during: {persistence['mean_effect_during']:.2f}")
print(f" Mean effect post: {persistence['mean_effect_post']:.2f}")
print(
f" Persistence ratio: {persistence['persistence_ratio']:.3f} ({persistence['persistence_ratio'] * 100:.1f}%)"
)
print(f" Total effect during: {persistence['total_effect_during']:.2f}")
print(f" Total effect post: {persistence['total_effect_post']:.2f}")
Accessing results programmatically:
Mean effect during: 24.53
Mean effect post: 6.36
Persistence ratio: 0.259 (25.9%)
Total effect during: 294.37
Total effect post: 356.37
Sensitivity checks in the pipeline#
Up to this point, we have estimated an effect and summarized how much of it persists. Sensitivity checks ask two follow-up questions before we get too comfortable with that story:
Check |
Question it answers |
Reassuring pattern |
|---|---|---|
|
If we move the intervention window into the pre-treatment period, do we still recover an effect this large? |
Fake intervention windows look much smaller than the observed effect. |
|
How much of the estimated effect remains after treatment ends? |
The persistence ratio matches the carryover story you want to tell. |
For fixed-period ITS, a placebo fold has to shift the whole intervention window, not just the start date. If the real campaign lasted 12 weeks, each fake campaign should also last 12 weeks so the placebo problem matches the real one. The code below handles that with a small experiment_factory built on SensitivityAnalysis.default_for(cp.InterruptedTimeSeries).
We use four placebo folds here because one or two can give a noisy impression of ordinary pre-treatment variation, while many more folds slow the notebook and leave less pre-period data for each fake intervention.
The pipeline in the next cell runs once. The cells after that only interpret the stored results.
Tip
The sampler settings below are intentionally smaller than the earlier model fit so the workflow stays quick to rerun in a notebook. For substantive analyses, increase draws, tune steps, and placebo folds.
from IPython.utils.capture import capture_output
fit_sample_kwargs = {
"draws": 300,
"tune": 300,
"chains": 2,
"cores": 1,
"progressbar": False,
"random_seed": seed,
"compute_convergence_checks": False,
}
placebo_sample_kwargs = {
"draws": 100,
"tune": 100,
"chains": 2,
"cores": 1,
"progressbar": False,
"random_seed": seed,
"compute_convergence_checks": False,
}
n_placebo_folds = 4
intervention_length = treatment_end_time - treatment_time
time_step = df.index[1] - df.index[0]
formula = "y ~ 1 + t + C(month, levels=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])"
def make_placebo_its(data, pseudo_treatment_time):
# Shift the full intervention window for fixed-period ITS placebo folds.
return cp.InterruptedTimeSeries(
data,
treatment_time=pseudo_treatment_time,
treatment_end_time=pseudo_treatment_time + intervention_length - time_step,
formula=formula,
model=cp.pymc_models.LinearRegression(sample_kwargs=fit_sample_kwargs),
)
default_sensitivity = cp.SensitivityAnalysis.default_for(cp.InterruptedTimeSeries)
for check in default_sensitivity.checks:
if isinstance(check, cp.checks.PlaceboInTime):
check.n_folds = n_placebo_folds
check.sample_kwargs.update(placebo_sample_kwargs)
check.experiment_factory = make_placebo_its
its_sensitivity = cp.SensitivityAnalysis(
checks=[*default_sensitivity.checks, cp.checks.PersistenceCheck()]
)
with capture_output() as sensitivity_run:
pipeline_result = cp.Pipeline(
data=df,
steps=[
cp.EstimateEffect(
method=cp.InterruptedTimeSeries,
treatment_time=treatment_time,
treatment_end_time=treatment_end_time,
formula=formula,
model=cp.pymc_models.LinearRegression(sample_kwargs=fit_sample_kwargs),
),
its_sensitivity,
],
).run()
sensitivity_results = {
check_result.check_name: check_result
for check_result in pipeline_result.sensitivity_results
}
placebo_result = sensitivity_results["PlaceboInTime"]
persistence_result = sensitivity_results["PersistenceCheck"]
Initializing NUTS using jitter+adapt_diag...
Sequential sampling (2 chains in 1 job)
NUTS: [beta, y_hat_sigma]
Sampling 2 chains for 300 tune and 300 draw iterations (600 + 600 draws total) took 2 seconds.
Sampling: [beta, y_hat, y_hat_sigma]
Sampling: [y_hat]
Sampling: [y_hat]
Sampling: [y_hat]
Sampling: [y_hat]
Initializing NUTS using jitter+adapt_diag...
Sequential sampling (2 chains in 1 job)
NUTS: [beta, y_hat_sigma]
Sampling 2 chains for 300 tune and 300 draw iterations (600 + 600 draws total) took 4 seconds.
Sampling: [beta, y_hat, y_hat_sigma]
Sampling: [y_hat]
Sampling: [y_hat]
Sampling: [y_hat]
Sampling: [y_hat]
Initializing NUTS using jitter+adapt_diag...
Sequential sampling (2 chains in 1 job)
NUTS: [beta, y_hat_sigma]
Sampling 2 chains for 300 tune and 300 draw iterations (600 + 600 draws total) took 3 seconds.
Sampling: [beta, y_hat, y_hat_sigma]
Sampling: [y_hat]
Sampling: [y_hat]
Sampling: [y_hat]
Sampling: [y_hat]
Initializing NUTS using jitter+adapt_diag...
Sequential sampling (2 chains in 1 job)
NUTS: [beta, y_hat_sigma]
Sampling 2 chains for 300 tune and 300 draw iterations (600 + 600 draws total) took 4 seconds.
Sampling: [beta, y_hat, y_hat_sigma]
Sampling: [y_hat]
Sampling: [y_hat]
Sampling: [y_hat]
Sampling: [y_hat]
Initializing NUTS using jitter+adapt_diag...
Sequential sampling (2 chains in 1 job)
NUTS: [beta, y_hat_sigma]
Sampling 2 chains for 300 tune and 300 draw iterations (600 + 600 draws total) took 2 seconds.
Sampling: [beta, y_hat, y_hat_sigma]
Sampling: [y_hat]
Sampling: [y_hat]
Sampling: [y_hat]
Sampling: [y_hat]
Initializing NUTS using jitter+adapt_diag...
Sequential sampling (2 chains in 1 job)
NUTS: [mu_status_quo, tau_status_quo, fold_z]
Sampling 2 chains for 100 tune and 100 draw iterations (200 + 200 draws total) took 2 seconds.
Sampling: [theta_new]
from IPython.display import Markdown, display
display(
Markdown(
f"""
The pipeline above ran once. The cells below reuse `pipeline_result.sensitivity_results`, so the remaining sections interpret stored results rather than rerunning the checks.
- `PlaceboInTime` asks whether fake pre-treatment windows can mimic the observed effect.
- `PersistenceCheck` asks how much of the estimated effect remains after treatment ends.
- Default ITS checks: `{[type(check).__name__ for check in default_sensitivity.checks]}`
- Checks run in this example: `{list(sensitivity_results)}`
- Configured placebo folds in this notebook: `{n_placebo_folds}`
- Why four? More folds give a better sense of ordinary pre-treatment variation, but each extra fold requires another full refit.
- If you want to inspect the raw sampler logs, run `sensitivity_run.show()` in a new cell.
"""
)
)
The pipeline above ran once. The cells below reuse pipeline_result.sensitivity_results, so the remaining sections interpret stored results rather than rerunning the checks.
PlaceboInTimeasks whether fake pre-treatment windows can mimic the observed effect.PersistenceCheckasks how much of the estimated effect remains after treatment ends.Default ITS checks:
['PlaceboInTime']Checks run in this example:
['PlaceboInTime', 'PersistenceCheck']Configured placebo folds in this notebook:
4Why four? More folds give a better sense of ordinary pre-treatment variation, but each extra fold requires another full refit.
If you want to inspect the raw sampler logs, run
sensitivity_run.show()in a new cell.
PlaceboInTime#
Start with the intuition: pretend the campaign happened earlier, entirely inside the pre-treatment period, and fit the same ITS model again. If those fake campaigns regularly produce effects close to the observed one, then the original effect is less distinctive.
Because this is a fixed-period ITS, each placebo fold shifts the whole 12-week intervention window backward. That keeps the placebo problem comparable to the real one: same duration, same model, different timing.
In this notebook we use four folds so you can see several fake intervention windows rather than only one or two. Each new fold starts one intervention-length earlier than the next, and each row in the table below corresponds to one fake intervention window.
placebo_fold_summary = pd.DataFrame(
[
{
"fold": fold.fold,
"pseudo_treatment_time": fold.pseudo_treatment_time,
"placebo_mean": fold.fold_mean,
"placebo_sd": fold.fold_sd,
}
for fold in placebo_result.metadata["fold_results"]
]
)
p_outside = placebo_result.metadata["p_effect_outside_null"]
actual_cumulative_impact = placebo_result.metadata["actual_cumulative_mean"]
placebo_fold_summary["pct_of_observed_effect"] = (
100 * placebo_fold_summary["placebo_mean"].abs() / abs(actual_cumulative_impact)
)
max_placebo_pct = placebo_fold_summary["pct_of_observed_effect"].max()
placebo_interpretation = (
"supports the original ITS effect"
if placebo_result.passed
else "does not clearly support the original ITS effect"
)
display(
Markdown(
f"""
`PlaceboInTime` asks whether we would recover effects of a similar size if we pretended the intervention started earlier in the pre-period. Here, the observed cumulative impact is `{actual_cumulative_impact:.2f}`.
How to read the table:
- `pseudo_treatment_time`: when the fake intervention starts
- `placebo_mean`: the estimated cumulative effect under that fake intervention
- `placebo_sd`: the uncertainty for that fake intervention
- `pct_of_observed_effect`: how large the placebo estimate is **in absolute value** relative to the observed effect
Some placebo folds are negative in this example. That is not a problem by itself. The key question is whether placebo effects, in either direction, are comparable in magnitude to the observed effect.
`p_effect_outside_null` is a compact summary of how separated the observed effect is from the placebo-based null distribution. In plain language, it is the estimated probability that the real cumulative effect is more extreme than what these fake intervention windows usually generate. It is not a frequentist p-value; it is a Bayesian separation score.
In this example, the largest placebo estimate in absolute value is only about `{max_placebo_pct:.1f}%` of the observed effect. The separation score is `{p_outside:.3f}`, and `placebo_result.passed` is `{placebo_result.passed}`, so this check **{placebo_interpretation}**.
"""
)
)
display(
placebo_fold_summary.round(
{"placebo_mean": 2, "placebo_sd": 2, "pct_of_observed_effect": 1}
)
)
PlaceboInTime asks whether we would recover effects of a similar size if we pretended the intervention started earlier in the pre-period. Here, the observed cumulative impact is 643.94.
How to read the table:
pseudo_treatment_time: when the fake intervention startsplacebo_mean: the estimated cumulative effect under that fake interventionplacebo_sd: the uncertainty for that fake interventionpct_of_observed_effect: how large the placebo estimate is in absolute value relative to the observed effect
Some placebo folds are negative in this example. That is not a problem by itself. The key question is whether placebo effects, in either direction, are comparable in magnitude to the observed effect.
p_effect_outside_null is a compact summary of how separated the observed effect is from the placebo-based null distribution. In plain language, it is the estimated probability that the real cumulative effect is more extreme than what these fake intervention windows usually generate. It is not a frequentist p-value; it is a Bayesian separation score.
In this example, the largest placebo estimate in absolute value is only about 34.9% of the observed effect. The separation score is 0.990, and placebo_result.passed is True, so this check supports the original ITS effect.
| fold | pseudo_treatment_time | placebo_mean | placebo_sd | pct_of_observed_effect | |
|---|---|---|---|---|---|
| 0 | 1 | 2022-10-16 | -224.70 | 347.34 | 34.9 |
| 1 | 2 | 2023-01-08 | -217.50 | 289.31 | 33.8 |
| 2 | 3 | 2023-04-02 | 115.61 | 325.26 | 18.0 |
| 3 | 4 | 2023-06-25 | 76.48 | 39.27 | 11.9 |
PersistenceCheck#
PersistenceCheck asks a different question from PlaceboInTime: not “could this be a spurious break?” but “how much of the effect remains after treatment ends?”
This is the same substantive persistence question we already explored with period="comparison" and analyze_persistence(). The advantage here is that persistence now appears inside the same pipeline as the placebo check, so both diagnostics can be reported together.
The most important quantity is the persistence ratio, which compares the average post-intervention effect with the average during-intervention effect. A value near 0 means the effect fades quickly. A value near 1 means the effect is roughly as strong after treatment ends as it was during treatment.
persistence = persistence_result.metadata["persistence"]
persistence_ratio = persistence["persistence_ratio"]
mean_effect_during = persistence["mean_effect_during"]
mean_effect_post = persistence["mean_effect_post"]
intervention_weeks = int(
((df.index >= treatment_time) & (df.index < treatment_end_time)).sum()
)
post_weeks = int((df.index >= treatment_end_time).sum())
persistence_summary = pd.DataFrame(
[
{
"period": "intervention",
"weeks_in_period": intervention_weeks,
"mean_effect": mean_effect_during,
"total_effect": persistence["total_effect_during"],
},
{
"period": "post_intervention",
"weeks_in_period": post_weeks,
"mean_effect": mean_effect_post,
"total_effect": persistence["total_effect_post"],
},
]
)
if persistence_ratio < 0.1:
persistence_interpretation = (
"little evidence that the effect persists once treatment ends"
)
elif persistence_ratio < 0.5:
persistence_interpretation = (
"partial persistence with substantial decay after treatment ends"
)
elif persistence_ratio < 1.0:
persistence_interpretation = (
"meaningful persistence, although the effect is smaller after treatment ends"
)
else:
persistence_interpretation = "a post-treatment effect that is at least as large as the during-treatment effect"
display(
Markdown(
f"""
`PersistenceCheck` compares the intervention-period effect with the post-intervention effect. The mean effect falls from `{mean_effect_during:.2f}` during treatment to `{mean_effect_post:.2f}` afterward. The persistence ratio is `{persistence_ratio:.3f}`, which means the average post-intervention effect is about `{100 * persistence_ratio:.1f}%` of the average during-intervention effect. In this example, that suggests **{persistence_interpretation}**.
How to read the table:
- `weeks_in_period`: how long that period lasts
- `mean_effect`: the average effect per week in that period
- `total_effect`: the cumulative effect summed over the whole period
The post-intervention total can be larger than the intervention total even when the post-intervention mean is smaller, simply because the post period lasts much longer. For interpretation, the ratio and the period means are usually more informative than comparing totals alone.
Read this alongside `PlaceboInTime`: the placebo result asks whether the estimated effect looks distinctive, while the persistence ratio asks how much of that effect remains once treatment ends.
"""
)
)
display(persistence_summary.round(3))
PersistenceCheck compares the intervention-period effect with the post-intervention effect. The mean effect falls from 24.43 during treatment to 6.26 afterward. The persistence ratio is 0.256, which means the average post-intervention effect is about 25.6% of the average during-intervention effect. In this example, that suggests partial persistence with substantial decay after treatment ends.
How to read the table:
weeks_in_period: how long that period lastsmean_effect: the average effect per week in that periodtotal_effect: the cumulative effect summed over the whole period
The post-intervention total can be larger than the intervention total even when the post-intervention mean is smaller, simply because the post period lasts much longer. For interpretation, the ratio and the period means are usually more informative than comparing totals alone.
Read this alongside PlaceboInTime: the placebo result asks whether the estimated effect looks distinctive, while the persistence ratio asks how much of that effect remains once treatment ends.
| period | weeks_in_period | mean_effect | total_effect | |
|---|---|---|---|---|
| 0 | intervention | 12 | 24.427 | 293.120 |
| 1 | post_intervention | 56 | 6.265 | 350.817 |
What to do when a check looks weak#
Treat a weak check as a cue to slow down and narrow your claims, not as something to hide.
If
PlaceboInTimereports large placebo effects, or most folds are skipped, the intervention effect may be hard to distinguish from ordinary pre-treatment fluctuation. Extend the pre-period if you can, revisit seasonality or trend terms, and report the estimate as fragile rather than definitive.If
PersistenceCheckshows a ratio near zero, the effect likely fades quickly after treatment ends. That can still be a useful result, but it argues against assuming long carryover.If the two diagnostics disagree, treat that as information. Report the disagreement and explain which assumptions or domain mechanisms make the estimate plausible.
For broader interpretation guidance across methods, see Sensitivity Checks in Pipeline Workflows.
Summary#
Here are the main takeaways from this notebook:
Three periods: Adding
treatment_end_timelets fixed-period ITS separate the pre-intervention, intervention, and post-intervention periods.Counterfactual thinking: The core comparison is always the observed series versus the no-treatment counterfactual learned from the pre-intervention pattern.
Persistence: In this example, the post-intervention effect is smaller than the intervention-period effect, which is what partial carryover looks like.
Quick read vs structured output:
period="comparison"gives the quickest persistence interpretation, whileanalyze_persistence()returns the same story in a more programmatic form.Placebo windows:
PlaceboInTimeasks whether similarly large effects appear when the same intervention window is moved into the pre-treatment period.Different sensitivity questions:
PlaceboInTimeasks whether the estimated effect looks distinctive;PersistenceCheckasks how much of it remains after treatment ends.Fragility matters: Weak or conflicting checks do not automatically invalidate the analysis, but they should lead to narrower and more cautious claims.
Reflection prompt: In your own domain, what would count as a believable placebo window, and how long would you expect an effect to last after treatment ends? For example, a retailer might ask whether a seasonal promotion leaves a short post-sale halo, a hospital might ask whether a staffing intervention changes outcomes only during rollout or afterward, and a product team might ask whether a feature launch creates temporary buzz or durable behavior change.
For a broader overview of check selection and interpretation, see Sensitivity Checks in Pipeline Workflows.
Optional Appendix: Coefficient table#
If you have the optional maketables package installed, you can render a formatted coefficient table for the base ITS fit. This is supplementary to the effect and sensitivity summaries above.
try:
from maketables import ETable
except ImportError:
print("Install `maketables` to render the formatted coefficient table.")
else:
ETable(result, coef_fmt="b:.3f")
Install `maketables` to render the formatted coefficient table.