Inference and Summarization

Methods for summarization and inference of estimators in the CausalELM package

CausalELM.summarizeFunction
summarize(study, mean_effect)

Return a summary from an event study.

Examples

julia> X₀, Y₀, X₁, Y₁ =  rand(100, 5), rand(100), rand(10, 5), rand(10)
julia> m1 = EventStudy(X₀, Y₀, X₁, Y₁)
julia> estimatetreatmenteffect!(m1)
[0.25714308]
julia> summarize(m1)
{"Task" => "Regression", "Regularized" => true, "Activation Function" => relu, 
"Validation Metric" => "mse","Number of Neurons" => 2, 
"Number of Neurons in Approximator" => 10, "β" => [0.25714308], 
"Causal Effect" => -3.9101138, "Standard Error" => 1.903434356, "p-value" = 0.00123356}
source
summarize(g)

Return a summary from a G-Computation estimator.

Examples

julia> X, Y, T =  rand(100, 5), rand(100), [rand()<0.4 for i in 1:100]
julia> m1 = GComputation(X, Y, T)
julia> estimatetreatmenteffect!(m1)
[0.3100468253]
julia> summarize(m1)
{"Task" => "Regression", "Quantity of Interest" => "ATE", Regularized" => "true", 
"Activation Function" => "relu", "Time Series/Panel Data" => "false", 
"Validation Metric" => "mse","Number of Neurons" => "5", 
"Number of Neurons in Approximator" => "10", "β" => "[0.3100468253]",
"Causal Effect: 0.00589761, "Standard Error" => 5.12900734, "p-value" => 0.479011245} 
source
summarize(dre, n)

Return a summary from a doubly robust estimator.

Examples

julia> X, Y, T =  rand(100, 5), rand(100), [rand()<0.4 for i in 1:100]
julia> m1 = DoublyRobust(X, X, Y, T)
julia> estimatetreatmenteffect!(m1)
[0.5804032956]
julia> summarize(m1)
{"Task" => "Regression", "Quantity of Interest" => "ATE", Regularized" => "true", 
"Activation Function" => "relu", "Validation Metric" => "mse", "Number of Neurons" => "5", 
"Number of Neurons in Approximator" => "10", "Causal Effect" = 0.5804032956, 
"Standard Error" => 2.129400324, "p-value" => 0.0008342356}
source
summarize(m, n)

Return a summary from a metalearner.

Examples

julia> X, Y, T =  rand(100, 5), rand(100), [rand()<0.4 for i in 1:100]
julia> m1 = SLearner(X, Y, T)
julia> estimatecate!(m1)
[0.20729633391630697, 0.20729633391630697, 0.20729633391630692, 0.20729633391630697, 
0.20729633391630697, 0.20729633391630697, 0.20729633391630697, 0.20729633391630703, 
0.20729633391630697, 0.20729633391630697  …  0.20729633391630703, 0.20729633391630697, 
0.20729633391630692, 0.20729633391630703, 0.20729633391630697, 0.20729633391630697, 
0.20729633391630692, 0.20729633391630697, 0.20729633391630697, 0.20729633391630697]
julia> summarise(m1)
{"Task" => "Regression", Regularized" => "true", "Activation Function" => "relu", 
"Time Series/Panel Data" => "false", "Validation Metric" => "mse", 
"Number of Neurons" => "5", "Number of Neurons in Approximator" => "10", 
"β" => "[0.3100468253]", "Causal Effect: [0.20729633391630697, 0.20729633391630697, 
0.20729633391630692, 0.20729633391630697, 0.20729633391630697, 0.20729633391630697, 
0.20729633391630697, 0.20729633391630703, 0.20729633391630697, 0.20729633391630697  …  
0.20729633391630703, 0.20729633391630697, 0.20729633391630692, 0.20729633391630703, 
0.20729633391630697, 0.20729633391630697, 0.20729633391630692, 0.20729633391630697, 
0.20729633391630697, 0.20729633391630697], "Standard Error" => 5.3121435085, 
"p-value" => 0.0632454855}
source
CausalELM.Inference.quantitiesofinterestFunction
quantitiesofinterest(model, n)

Generate a p-value and standard error through randomization inference

This method generates a null distribution of treatment effects by reestimating treatment effects from permutations of the treatment vector and estimates a p-value and standard from the generated distribution.

Note that lowering the number of iterations increases the probability of failing to reject the null hypothesis.

For a primer on randomization inference see: https://www.mattblackwell.org/files/teaching/s05-fisher.pdf

Examples

julia> x, y, t = rand(100, 5), rand(1:100, 100, 1), [rand()<0.4 for i in 1:100]
julia> g_computer = GComputation(x, y, t)
julia> estimatecausaleffect!(g_computer)
julia> quantitiesofinterest(g_computer, 1000)
(0.114, 6.953133617011371)
source
quantitiesofinterest(model, nsplits)

Generate a p-value and standard error through randomization inference

This method generates a null distribution of treatment effects by reestimating treatment effects from permutations of the treatment vector and estimates a p-value and standard from the generated distribution. Randomization for event studies is done by creating time splits at even intervals and reestimating the causal effect.

Note that lowering the number of iterations increases the probability of failing to reject the null hypothesis.

For a primer on randomization inference see: https://www.mattblackwell.org/files/teaching/s05-fisher.pdf

Examples

julia> x₀, y₀, x₁, y₁ = rand(1:100, 100, 5), rand(100), rand(10, 5), rand(10)
julia> event_study = EventStudy(x₀, y₀, x₁, y₁)
julia> estimatecausaleffect!(event_study)
julia> quantitiesofinterest(event_study, 10)
(0.0, 0.07703275541001667)
source
CausalELM.Inference.generatenulldistributionFunction
generatenulldistribution(e, n)

Generate a null distribution for the treatment effect of G-computation, doubly robust estimation, or metalearning.

This method estimates the same model that is provided using random permutations of the treatment assignment to generate a vector of estimated effects under different treatment regimes. When e is a metalearner the null statistic is the difference is the ATE.

Note that lowering the number of iterations increases the probability of failing to reject the null hypothesis.

Examples

julia> x, y, t = rand(100, 5), rand(1:100, 100, 1), [rand()<0.4 for i in 1:100]
julia> g_computer = GComputation(x, y, t)
julia> estimatecausaleffect!(g_computer)
julia> generatenulldistribution(g_computer, 500)
[0.016297180690693656, 0.0635928694685571, 0.20004144093635673, 0.505893866040335, 
0.5130594630907543, 0.5432486130493388, 0.6181727442724846, 0.61838399963459, 
0.7038981488009489, 0.7043407710415689  …  21.909186142780246, 21.960498059428854, 
21.988553083790023, 22.285403459215363, 22.613625375395973, 23.382102081355548, 
23.52056245175936, 24.739658523175912, 25.30523686137909, 28.07474553316176]
source
generatenulldistribution(e, n, mean_effect)

Generate a null distribution for the treatment effect in an event study design. By default, this method generates a null distribution of mean differences. To generate a null distribution of cummulative differences, set the mean_effect argument to false.

Instead of randomizing the assignment of units to the treamtent or control group, this method generates the null distribution by reestimating the event study with the intervention set to n splits at even intervals within the total study duration.

Note that lowering the number of iterations increases the probability of failing to reject the null hypothesis.

For a primer on randomization inference see: https://www.mattblackwell.org/files/teaching/s05-fisher.pdf

Examples

julia> x₀, y₀, x₁, y₁ = rand(1:100, 100, 5), rand(100), rand(10, 5), rand(10)
julia> event_study = EventStudy(x₀, y₀, x₁, y₁)
julia> estimatecausaleffect!(event_study)
julia> generatenulldistribution(event_study, 10)
[-0.5012456678829079, -0.33790650529972194, -0.2534340182760628, -0.21030239864895905, 
-0.11672915615117885, -0.08149441936166794, -0.0685134758182695, -0.06217013151235991, 
-0.05905529159312335, -0.04927743270606937]
source