Disclaimer: The purpose of the Open Case Studies project is to demonstrate the use of various data science methods, tools, and software in the context of messy, real-world data. A given case study does not cover all aspects of the research process, is not claiming to be the most appropriate way to analyze a given data set, and should not be used in the context of making policy decisions without external consultation from scientific experts.

This work is licensed under the Creative Commons Attribution-NonCommercial 3.0 (CC BY-NC 3.0) United States License.

To cite this case study please use:

Wright, Carrie and Ontiveros, Michael and Meng, Qier and Jager, Leah and Taub, Margaret and Hicks, Stephanie. (2020). https://github.com//opencasestudies/ocs-bp-RTC-analysis. Influence of Multicollinearity on Measured Impact of Right-to-Carry Gun Laws (Version v1.0.0).

To access the GitHub repository for this case study see here: https://github.com//opencasestudies/ocs-bp-RTC-analysis.

You may also access and download the data using our OCSdata package. To learn more about this package including examples, see this link. Here is how you would install this package:

install.packages("OCSdata")

This case study is part of a series of public health case studies for the Bloomberg American Health Initiative.

The total reading time for this case study is calculated via koRpus and shown below:

Reading Time	Method
58 minutes	koRpus

Readability Score:

A readability index estimates the reading difficulty level of a particular text. Flesch-Kincaid, FORCAST, and SMOG are three common readability indices that were calculated for this case study via koRpus. These indices provide an estimation of the minimum reading level required to comprehend this case study by grade and age.

Text language: en

index	grade	age
Flesch-Kincaid	11	16
FORCAST	10	15
SMOG	13	18

Please help us by filling out our survey.

Motivation

This case study will introduce the topic of multicollinearity, which occurs in regression when one or more independent variables can be predicted by other independent variables.

We will do so by showcasing a real world example where multicollinearity in part resulted in historically controversial and conflicting findings about the influence of the adoption of right-to-carry (RTC) concealed handgun laws on violent crime rates in the United States.

We will focus on two articles:

The first analysis by Mustard and Lott published in 1996 suggests that RTC laws reduce violent crime. Lott authored a book extending these findings in 1998 called More Guns, Less Crime.

[source]

The second analysis is a recent article by Donohue, et al. published in 2017 that suggests that RTC laws increase violent crime. Donohue has also published previous articles with titles such as Shooting down the “More Guns, Less Crime” Hypothesis.

[source]

This has been a controversial topic as many other analyses also produced conflicting results. See here for a list of studies.

The Donohue, et al. article discusses how there are many other important methodological aspects besides multicollinearity (which occurs when predictor or input variables are highly related in a regression analysis) that could account for the historically conflicting results in these previous manuscripts.

In fact, nearly every aspect of the data analysis process was different between the Donohue, et al. and Mustard and Lott analyses.

However, we will focus particularly on multicollinearity and how it can influence the results we get from linear regression. Specifically, this analysis will demonstrate how methodological details can be critically influential for our overall conclusions and can result in important policy related consequences. The Donohue, et al. article will provide the motivation and illustration.

John J. Donohue et al., Right‐to‐Carry Laws and Violent Crime: A Comprehensive Assessment Using Panel Data and a State‐Level Synthetic Control Analysis. Journal of Empirical Legal Studies, 16,2 (2019).

David B. Mustard & John Lott. Crime, Deterrence, and Right-to-Carry Concealed Handguns. Coase-Sandor Institute for Law & Economics Working Paper No. 41, (1996).

Before we leave this section, we provide a high-level overview of what variables were (or were not) included in the Donohue, Aneja and Weber (DAW) paper and the Mustard and Lott (ML) paper:

[source]

*ML is abbreviated as LM in the source article

Note: We are not attempting to re-create the analyses from the original authors. Instead, we aim to use a subset of the listed explanatory variables in this case study to demonstrate multicollinearity. These variables will be consistent for both analyses that we will perform, with the exception that one analysis will have 6 demographic variables as in the analysis in the Donohue, et al. article and the other will have 36 demographic variables, grouping individuals into more specific categories, as in the analysis in the Mustard and Lott article.

Main Question

Our main question:

What is the effect of multicollinearity on coefficient estimates from linear regression models when analyzing right to carry laws and violence rates?

Specifically, we will consider the two ways to define the demographic variables (as described above) and investigate how the inclusion of different numbers of age groups influences the results of an analysis of right to carry laws and violence rates.

Learning Objectives

The skills, methods, and concepts that students will be familiar with by the end of this case study are:

Data Science Learning Objectives:

Create correlation scatterplots and heatmaps (GGally, ggcorrplot)
Create interactive tables (DT)
Sample subsets of data (rsample)
Combine multiple plots (cowplot)
Create data visualizations with equations and text(ggplot2 and latex2exp)

Statistical Learning Objectives:

Understand what multicollinearity is and how it can influence linear regression coefficients
Recognize signs for the presence of multicollinearity and determine its severity
Illustrate the difference between multicollinearity and correlation
Implement panel regression analysis in R (plm)
Relate variance inflation factors (VIFs) to multicollinearity and calculate VIFs in R (car)

To see another case study about how the original raw data was imported and wrangled please see here.

We will especially focus on using packages and functions from the tidyverse, such as dplyr and ggplot2. The tidyverse is a library of packages created by RStudio. While some students may be familiar with previous R programming packages, these packages make data science in R especially legible and intuitive.

Context

So what exactly is a right-to-carry law?

It is a law that specifies if and how citizens are allowed to have a firearm on their person or nearby (for example, in a citizen’s car) in public. In this discussion, we will use the National Rifle Association (NRA) terminology. Please keep in mind that there are other terms that people use.

The Second Amendment to the United States Constitution guarantees the right to “keep and bear arms”. The amendment was ratified in 1791 as part of the Bill of Rights.

[source]

However, there are no federal laws about carrying firearms in public.

These laws are created and enforced at the US state level. States vary greatly in their laws about the right to carry firearms. Some require extensive effort to obtain a permit to legally carry a firearm, while other states require very minimal effort to do so. An increasing number of states do not require permits at all.

Click here for more information on history of right-to-carry policies in the US.

According to the Wikipedia entry about the history of right-to-carry policies in the United States:

Public perception on concealed carry vs open carry has largely flipped. In the early days of the United States, open carrying of firearms, long guns and revolvers was a common and well-accepted practice. Seeing guns carried openly was not considered to be any cause for alarm. Therefore, anyone who would carry a firearm but attempt to conceal it was considered to have something to hide, and presumed to be a criminal. For this reason, concealed carry was denounced as a detestable practice in the early days of the United States.

Concealed weapons bans were passed in Kentucky and Louisiana in 1813. (In those days open carry of weapons for self-defense was considered acceptable; concealed carry was denounced as the practice of criminals.) By 1859, Indiana, Tennessee, Virginia, Alabama, and Ohio had followed suit. By the end of the nineteenth century, similar laws were passed in places such as Texas, Florida, and Oklahoma, which protected some gun rights in their state constitutions. Before the mid 1900s, most U.S. states had passed concealed carry laws rather than banning weapons completely. Until the late 1990s, many Southern states were either “No-Issue” or “Restrictive May-Issue”. Since then, these states have largely enacted “Shall-Issue” licensing laws, with numerous states legalizing “Unrestricted concealed carry”.

There are five broad categories of right-to-carry laws according to the NRA:

[source]

You can see that no state in the US currently (this map is from 2020) has a “Rights Infringed/Non-Issue” law (the gray category) – meaning that all 50 states in the US allow the right to carry firearms at least in some way. However the level of restrictions is dramatically different from one state to another. For comparison purposes, many authors use the terms “shall issue”, “shall issue with discretion”, “no permit required”, “may issue”, and “non-issue” to describe these categories instead of the NRA terminology.

Click here for more information about how restrictions vary from one state to another.

There is variation from state to state even within the same general category:

For example here is an abridged version of the current carry laws in Idaho which is considered an “Unrestricted - no permit required” state:

State law … allows any resident of Idaho or a current member of the armed forces of the United States to carry a concealed handgun without a license to carry, provided the person is over 18 years old and not disqualified from being issued a license to carry concealed weapons under state law. An amendment to state law that takes effect on July 1, 2020 changes the reference in the above law from “a resident of Idaho” to “any citizen of the United States.”

And here are is an abridged version of the current carry laws in Arizona which is also considered an “Unrestricted - no permit required” state:

Any person 21 years of age or older, who is not prohibited possessor, may carry a weapon openly or concealed without the need for a license…

Notice that citizens in Idaho only need to be 18 to carry a firearm, whereas they must be 21 in Arizona.

Limitations

There are some important considerations regarding this data analysis to keep in mind:

We do not use all of the data used by either the Mustard and Lott or Donohue, et al. analyses, nor do we perform the same analysis as in each article. We instead perform a much simpler analysis with fewer variables for the purposes of illustration of the concept of multicollinearity and its influence on regression coefficients, not to reproduce either analysis.
Our analysis accounts for either the adoption or lack of adoption of a permissive right-to-carry law in each state, but does not account for differences in the level of permissiveness of the laws.

Recall that these are the categories of right to carry laws:

States with laws of the category rights restricted - very limited issue (red) are considered as not having a permissive right-to-carry law. Recall that no states currently have a rights infringed/non-issue law.

States of all other categories (shall issue, discretionary/reasonable issue, and no permit required, all shades of blue) are considered the same in our analysis, as having a permissive right-to-carry law.

Because our analysis is an oversimplification, the results presented here should not be used for determining policy changes; instead we suggest that users interested in such a determination consult with a specialist.
The inclusion of race as an explanatory variable in an epidemiological study can be useful in certain circumstances. However, there are limitations and issues around defining, determining, and reporting race, as well as in interpreting differences in public health outcomes by race. For more information on this topic, we have included a link to a paper on the use of race as a measure in epidemiology. We include race in this analysis to demonstrate and consider the limitations of what the previous papers have done to analyze the influence of RTC laws on violent crime, with a focus on multicollinearity. Thus in our analysis we have also defined race as was previously done in these papers. Furthermore, we want to point out that reporting analyses about crime with race as a variable can have very unexpected consequences and thus care should be taken. See here for suggestions. Any association between demographic variables (indicating the proportion of the population from specific race and age groups) and violent crime does not necessarily indicate that the two are linked causally, as aside from the issues presented in the article, this may instead indicate higher rates of police engagement with certain racial groups due to racial profling.

The ACLU defines racial profiling as:

“Racial Profiling” refers to the discriminatory practice by law enforcement officials of targeting individuals for suspicion of crime based on the individual’s race, ethnicity, religion or national origin.

We will begin by loading the packages that we will need:

library(here)
library(dplyr)
library(magrittr)
library(purrr)
library(tibble)
library(ggplot2)
library(naniar)
library(ggrepel)
library(plm)
library(broom)
library(GGally)
library(ggcorrplot)
library(rsample)
library(DT)
library(car)
library(stringr)
library(cowplot)
library(latex2exp)
library(OCSdata)

Packages used in this case study:

Package	Use in this case study
`here`	to easily load and save data
`dplyr`	to arrange/filter/select/compare specific subsets of the data
`magrittr`	to use the compound assignment pipe operator `%<>%`
`purrr`	to import the data in all the different excel and csv files efficiently
`tibble`	to create data objects that we can manipulate with `dplyr`/`stringr`/`tidyr`/`purrr`
`ggplot2`	to create plots
`naniar`	to quickly visualize missing data
`ggrepel`	to allow labels in figures not to overlap
`plm`	to work with panel data fitting fixed effects and linear regression models
`broom`	to create nicely formatted model output
`GGally`	to extend ggplot2 functionality to easily create more complex plots
`ggcorrplot`	to easily visualize a correlation matrix
`rsample`	to split our sample for the simulation analysis
`DT`	to create interactive and searchable tables
`car`	to calculate VIF values on linear model output
`stringr`	to manipulate the character strings within the data
`cowplot`	to allow plots to be combined
`latex2exp`	to convert latex math formulas to R’s plotmath expressions
`OCSdata`	to access and download OCS data files

The first time we use a function, we will use the :: to indicate which package we are using. Unless we have overlapping function names, this is not necessary, but we will include it here to be informative about where the functions we will use come from.

What are the data?

Below is a table from the Donohue, et al. paper that shows the data used in both analyses, where DAW stands for Donohue, et al. and LM stands for Mustard and Lott.

We will be using a subset of these variables, which are highlighted in green:

Data Import and Wrangling

See this case study for details about data import and wrangling. We include data sources here for completeness, but will start from processed data in this case study.

Demographic and population data

To obtain information about age, sex, race, and overall population size we will use US Census Bureau data, as was done in both of the articles. The census data is available for different time spans. Here are the links for the years used in our analysis. We will use data from 1977 to 2010.

Data	Link
years 1977 to 1979	link
years 1980 to 1989	link * county data was used for this decade which also has state information
years 1990 to 1999	link
years 2000 to 2010	link technical documentation

State FIPS codes

The data was downloaded from the US Census Bureau.

Police staffing data

The following data was downloaded from the Federal Bureau of Investigation.

Unemployment data

The following data was downloaded from the U.S. Bureau of Labor Statistics.

Poverty data

Extracted from Table 21 from US Census Bureau Poverty Data.

Violent crime

Violent crime data was obtained from here.

Right-to-carry data

This data is extracted from table in Donohue paper.

Here is the table from the Donohue paper that compares the data used in the analyses:

[source]

*ML is abbreviated as LM in the source article

We can see that only the percentage of males that were from age 15-39 of the race groups (black, white, and other) were used in the Donohue analysis, whereas the Mustard and Lott paper, individuals from ages 10 to over 65 were used.

The final products from our data wrangling which is described in another case study are two tibbles of data each with variables selected to be similar to those used in either the Donohue or Mustard and Lott analysis. The overarching idea of this case study is to compare the model results from these two datasets, which differ only in the demographic variables. This is an oversimplification of the actual differences between the datasets and approaches taken by Donohue and Lott/Mustard, but will be useful for illustrating the impact of multicollinearity on our modeling results, which is the main objective of this case study.

If you have trouble accessing the GitHub Repository, the “Wrangled_data.rda” data can be downloaded from here.

We will load this data now.

In our case, we downloaded this data and put it within a “wrangled” subdirectory of the “data” directory for our project. If you use an RStudio project, then you can use the here() function of the here package to make the path for loading this data simpler. The here package automatically starts looking for files based on where you have a .Rproj file which is created when you start a new RStudio project. We can specify that we want to look for the file within the “wrangled” subdirectory of the “data” directory within a directory where our .Rproj file is located by separating the name of the “data” directory and “wrangled” and the file name using commas.

Click here to see more about creating new projects in RStudio.

You can create a project by going to the File menu of RStudio like so:

You can also do so by clicking the project button:

See here to learn more about using RStudio projects.

load(file = here::here("data", "wrangled", "Wrangled_data.rda"))

You may also download the data files with the OCSdata package. If you prefer this method, use the following code chunk to download and load the data:

# install.packages("OCSdata")
library(OCSdata)
wrangled_rda("ocs-bp-RTC-analysis", outpath = getwd())
# This will save the wrangled data files in a "OCSdata/data/wrangled/" 
# sub-folder in your current working directory
load(file = here::here("OCSdata", "data", "wrangled", "Wrangled_data.rda"))
# this will load the data into R

We will check the dimensions of each tibble using the base dim() function:

dim(LOTT_DF)

[1] 1395   50

dim(DONOHUE_DF)

[1] 1395   20

As expected the Lott_DF is 30 columns larger, due to the 30 additional demographic variables. We can check those now as well.

LOTT_DF %>%
  colnames()

 [1] "YEAR"                           "STATE"                         
 [3] "Black_Female_10_to_19_years"    "Black_Female_20_to_29_years"   
 [5] "Black_Female_30_to_39_years"    "Black_Female_40_to_49_years"   
 [7] "Black_Female_50_to_64_years"    "Black_Female_65_years_and_over"
 [9] "Black_Male_10_to_19_years"      "Black_Male_20_to_29_years"     
[11] "Black_Male_30_to_39_years"      "Black_Male_40_to_49_years"     
[13] "Black_Male_50_to_64_years"      "Black_Male_65_years_and_over"  
[15] "Other_Female_10_to_19_years"    "Other_Female_20_to_29_years"   
[17] "Other_Female_30_to_39_years"    "Other_Female_40_to_49_years"   
[19] "Other_Female_50_to_64_years"    "Other_Female_65_years_and_over"
[21] "Other_Male_10_to_19_years"      "Other_Male_20_to_29_years"     
[23] "Other_Male_30_to_39_years"      "Other_Male_40_to_49_years"     
[25] "Other_Male_50_to_64_years"      "Other_Male_65_years_and_over"  
[27] "White_Female_10_to_19_years"    "White_Female_20_to_29_years"   
[29] "White_Female_30_to_39_years"    "White_Female_40_to_49_years"   
[31] "White_Female_50_to_64_years"    "White_Female_65_years_and_over"
[33] "White_Male_10_to_19_years"      "White_Male_20_to_29_years"     
[35] "White_Male_30_to_39_years"      "White_Male_40_to_49_years"     
[37] "White_Male_50_to_64_years"      "White_Male_65_years_and_over"  
[39] "Unemployment_rate"              "Poverty_rate"                  
[41] "Viol_crime_count"               "Population"                    
[43] "police_per_100k_lag"            "RTC_LAW_YEAR"                  
[45] "RTC_LAW"                        "TIME_0"                        
[47] "TIME_INF"                       "Viol_crime_rate_1k"            
[49] "Viol_crime_rate_1k_log"         "Population_log"

DONOHUE_DF %>%
  colnames()

 [1] "YEAR"                      "STATE"                    
 [3] "Black_Male_15_to_19_years" "Black_Male_20_to_39_years"
 [5] "Other_Male_15_to_19_years" "Other_Male_20_to_39_years"
 [7] "White_Male_15_to_19_years" "White_Male_20_to_39_years"
 [9] "Unemployment_rate"         "Poverty_rate"             
[11] "Viol_crime_count"          "Population"               
[13] "police_per_100k_lag"       "RTC_LAW_YEAR"             
[15] "RTC_LAW"                   "TIME_0"                   
[17] "TIME_INF"                  "Viol_crime_rate_1k"       
[19] "Viol_crime_rate_1k_log"    "Population_log"

Lastly, we will check that the YEAR values are the same, i.e., that the two tibbles contain data from the same set of years. We can use the setequal() function of the dplyr package to see if the values are the same.

setequal(
  DONOHUE_DF %>% distinct(YEAR),
  LOTT_DF %>% distinct(YEAR)
)

[1] TRUE

Data Exploration

Let’s do some quick visualizations to get a sense of our outcome of interest, the violent crime data.

First we will plot the rate of violent crime over time to get a sense of the general trend.

To do so we need to summarize the data for each year across all of the states. Thus we will use the group_by() function and the summarize() functions to calculate an overall number of violent crimes relative to the total population for each year. In fact, we will calculate the log of the number of violent crimes per 100,000 individuals, as this will make it easier to see the trend in the data. In addition, by looking at the rate per 100,000 individuals, we can more directly compare crime statistics across groups like states with different populations, as you will see below.

Then we will use the ggplot2 package to plot the data. The first step in creating a plot with this package is to use the ggplot() function and the aes() argument to specify what data should be plotted on the x-axis and what data should be plotted in on the y-axis. Then we select what type of plot we would like to make using one of the geom_*() functions. Please see this case study for more details.

We can use the scale_x_continuous() and scale_y_continuous() functions to modify the axis tick marks and their labels.

The labs() function can be used to add labels to the plot, while the theme() function allows for manipulation of the details of the labels, like size and angle.

All of these functions are part of the ggplot2 package.

DONOHUE_DF %>%
  group_by(YEAR) %>%
  summarize(
    Viol_crime_count = sum(Viol_crime_count),
    Population = sum(Population),
    .groups = "drop"
  ) %>%
  mutate(Viol_crime_rate_100k_log = log((Viol_crime_count * 100000) / Population)) %>%
  ggplot(aes(x = YEAR, y = Viol_crime_rate_100k_log)) +
  geom_line() +
  scale_x_continuous(
    breaks = seq(1980, 2010, by = 1),
    limits = c(1980, 2010),
    labels = c(seq(1980, 2010, by = 1))
  ) +
  scale_y_continuous(
    breaks = seq(5.75, 6.75, by = 0.25),
    limits = c(5.75, 6.75)
  ) +
  labs(
    title = "Crime rates fluctuate over time",
    x = "Year",
    y = "ln(violent crimes per 100,000 people)"
  ) +
  theme_minimal() +
  theme(axis.text.x = element_text(angle = 90))

Interesting! It appears that there was an overall national peak in violent crime in the early 1990s that has since then declined.

Now let’s take a look at each state.

We will use the ggrepel package to add text to the plot using the geom_text_repel() function. This is especially useful when there is a lot of text, as this function reduces the overlap of text labels. Again see this case study for more details on how to add labels to elements of plots.

DONOHUE_DF %>%
  mutate(Viol_crime_rate_100k_log = log((Viol_crime_count * 100000) / Population)) %>%
  ggplot(aes(x = YEAR, y = Viol_crime_rate_100k_log, color = STATE)) +
  geom_point(size = 0.5) +
  geom_line(aes(group = STATE),
    size = 0.5,
    show.legend = FALSE
  ) +
  geom_text_repel(
    data = DONOHUE_DF %>%
      mutate(Viol_crime_rate_100k_log = log((Viol_crime_count * 100000) / Population)) %>%
      filter(YEAR == last(YEAR)),
    aes(
      label = STATE,
      x = YEAR,
      y = Viol_crime_rate_100k_log
    ),
    size = 3,
    alpha = 1,
    nudge_x = 10,
    direction = "y",
    hjust = 1,
    vjust = 1,
    segment.size = 0.25,
    segment.alpha = 0.25,
    force = 1,
    max.iter = 9999
  ) +
  guides(color = FALSE) +
  scale_x_continuous(
    breaks = seq(1980, 2015, by = 1),
    limits = c(1980, 2015),
    labels = c(seq(1980, 2010, by = 1), rep("", 5))
  ) +
  scale_y_continuous(
    breaks = seq(3.5, 8.5, by = 0.5),
    limits = c(3.5, 8.5)
  ) +
  labs(
    title = "States have different levels of crime",
    x = "Year",
    y = "ln(violent crimes per 100,000 people)"
  ) +
  theme_minimal() +
  theme(axis.text.x = element_text(angle = 90))

It looks like the crime rates vary quite a bit from one state to another. Some states show increased crime over time while others show decreased crime.

Now let’s take a closer look at some of our other variables.

As we do this, we may start to run into issues with missing data for some of our variables, and it is always good to be aware of missing data in any data analysis. We can use the vis_miss() function of the naniar package to confirm that there are no missing values.

DONOHUE_DF %>%
  naniar::vis_miss()

Looks like no missing data!

LOTT_DF %>%
  naniar::vis_miss()

Same for the LOTT_DF.

We can use the skim() of the skimr package to get a better sense of the data. This also shows missingness, as well as standard deviations, spread, and means for our data. Also notice that there is a small histogram of each variable in the final column.

skimr::skim(DONOHUE_DF)

skimr::skim(LOTT_DF)

We can see from this function that we have the number of variables of the class types that we expect for each tibble. We can also see that the means of the variables that should be the same for each tibble are in fact the same. We can also tell that the values for the variables are in general what we would expect.

Data Analysis

Panel Analysis

In these datasets, we have what is called panel data, a special type of longitudinal data. Longitudinal data are data measurements taken over time. Panel data are data repeatedly measured for multiple panel members or individuals over time. This is in contrast with time series data, which measures one individual over time and cross sectional data, which measures multiple individuals at one point in time. In other words, panel data is a combination of both, with measurements for multiple individuals/units of observation over multiple time periods. In our case, we have measurements of violent crime and other variables for each state over many years. Therefore we are using measurements about the same states over time.

In a panel analysis there are $N$ individual panel members and $T$ time points.

There are two types of panels:
1. Balanced - At each time point ($T$), there are data points for each individual($N$).

Time Points ($T$)	Individuals ($N$)
1977	Nevada
1977	Alabama
1977	Kansas
1978	Nevada
1978	Alabama
1978	Kansas
1979	Nevada
1979	Alabama
1979	Kansas

Unbalanced - There may be data points missing for some individuals ($N$) at some time points ($T$).

Time Points ($T$)	Individuals ($N$)
1977	Nevada
1977	Alabama
1978	Nevada
1978	Alabama
1979	Nevada
1979	Alabama
1979	Kansas

Overall in a balanced panel, we have $n$ observations, where $n = N*T$.

In an unbalanced panel, the number of observations is less than $N*T$.

In our case we have:
$N$ = 45 states (in the data wrangling process we removed those who had adopted an RTC law before 1980)
$T$ = 31 years (1980 - 2010)

In every year we have measurements for each state (as we just saw above), thus our panel is balanced.

So, our total observations $n = 45*31$, thus $n$ = 1395.

We will be performing a panel linear regression model analysis.

In such an analysis we will model our data according to this generic model:

\[Y_{it}=β_{0}+β_{1}X_{1it}+...+β_{K}X_{Kit}+e_{it}\]

Where $i$ is the individual dimension (in our case individual states) and $t$ is the time dimension.

Some explanatory/independent variables or regressors $X_{it}$ will vary across individuals and time, while others will be fixed across the time of the study (or don’t change over time), while others still will be fixed across individuals but vary across time periods.

There are three general sub-types of panel regression analysis.

Overall, they assume that the different individuals are independent, however the same data for the same individual may be correlated across time.

The main difference between the three sub-types are the assumptions about unobserved differences between individuals.

If you are familiar with fixed and random effects in the context in the statistical literature the following econometric definitions will sound a bit different.

From the plm package vignette:

In the mixed models literature,…fixed effect indicates a parameter that is assumed constant, while random effects are parameters that vary randomly around zero according to a joint multivariate normal distribution.

…Having fixed effects in an econometric model has the meaning of allowing the intercept to vary with group, or time, or both, while the other parameters are generally still assumed to be homogeneous. Having random effects means having a group– (or time–, or both) specific component in the error term.

OK, so now that we know not to expect the typical mixed model definitions of fixed and random effects, let’s get back to the three sub-types of panel regression analysis:

independently pooled panels - assumes that there are no individual effects that are independent of time and also no effect of time on all the individuals. In other words, the independent variables are not correlated with any error term. This is essentially an ordinary least squares linear regression. In our setting, it would mean treating each year-state observation as unrelated to the others, which probably does not make sense, since we expect observations within a state to be related to one another. This type of panel regression makes the most assumptions and is therefore typically not used for panel data. In this case the model formulation is: \[Y_{it}=\beta_{0}+\beta_{1}x_{1it}+...+\beta_{K}X_{Kit} + e_{it}\] where the intercept $\beta_{0it}=\beta_0$for all $i,t$ and slope $\beta_{kit}=\beta_k$ for all $i,t$.
fixed effects - assumes that there are unknown or unobserved unique aspects about the individuals or heterogeneity among individuals $a_i$ that are not explained by the independent variables but influence the outcome variable of interest. They do not vary with time or in other words are fixed over time but may be correlated with independent variables $X_{it}$.

In this case the intercept can be different for each individual $\beta_{0i}$, but the other coefficients are assumed to be the same across all the individuals.

These individual $a_i$ effects can be correlated with the independent variables $X$. This model can be expressed as \[Y_{it}=\beta_{0}+\beta_{1}X_{1it}+...\beta_{K}X_{Kit}+ a_i +e_{it}\] or alternatively the individual effects can be absorbed into an individual-specific intercept term $\beta_{0i}$: \[Y_{it}=\beta_{0i}+\beta_{1}X_{1it}+...\beta_{k}X_{kit} +e_{it}\] This type of panel regression makes the fewest assumptions.

random effects - assumes that there are unknown or unobserved unique qualities about the individuals that influence the outcome variable of interest that are not correlated with the independent variables. Thus, the random effects model actually makes more assumptions than the fixed effect model. We will not consider this model here.

There is quite a lot of discussion in econometrics around which model is appropriate to use and when, but we are not going to dwell on this topic here, as our main focus in this case study is on multicollinearity. However, here are some additional references on these topics for further reading: See here and here and here for more information about these different models.

Donohue, et al.

OK! We are now ready to start analyzing our data!

In our case, we will be performing a fixed effect panel regression analysis, as we do in fact think that some of the unobserved qualities about the different states may be correlated with some of our independent variables. For example, the level of economic opportunity might be an unobserved feature about the states that influences violent crime rate and would be possibly correlated with poverty rate and unemployment. There are statistical tests for evaluating which model is the most appropriate, which are implemented in the plm package that we will be using.

To perform our analysis we will be using the plm package. This stands for Panel Linear Model.

We need to use a special type of data to use this package, called a pdata.frame which is short for panel data frame. This allows us to specify that we are using panel data and what the panel structure looks like, i.e., how the different observations are meant to be grouped together, both by state and by time.

We need to indicate which variable should be used to identify the individuals in our panel, and what variable should be used to identify the time periods in our panel. In our case the STATE variable identifies the individuals and the YEAR variable identifies the time periods.

We can specify this structure using the pdata.frame() function of the plm package, by using the index argument, where the individual variable is specified first followed by the time variable, like so: index=c("Individual_Variable_NAME", "Time_Period_Variable_NAME").

d_panel_DONOHUE <- pdata.frame(DONOHUE_DF, index = c("STATE", "YEAR"))

class(d_panel_DONOHUE)

[1] "pdata.frame" "data.frame"

slice_head(d_panel_DONOHUE, n = 3)

            YEAR  STATE Black_Male_15_to_19_years Black_Male_20_to_39_years
Alaska-1980 1980 Alaska                 0.1670456                 0.9933775
Alaska-1981 1981 Alaska                 0.1732299                 1.0028219
Alaska-1982 1982 Alaska                 0.1737069                 1.0204445
            Other_Male_15_to_19_years Other_Male_20_to_39_years
Alaska-1980                  1.129782                  2.963329
Alaska-1981                  1.124441                  2.974775
Alaska-1982                  1.069821                  3.015071
            White_Male_15_to_19_years White_Male_20_to_39_years
Alaska-1980                  3.627805                  18.28852
Alaska-1981                  3.558261                  18.12821
Alaska-1982                  3.391844                  18.10666
            Unemployment_rate Poverty_rate Viol_crime_count Population
Alaska-1980               9.6          9.6             1919     404680
Alaska-1981               9.4          9.0             2537     418519
Alaska-1982               9.9         10.6             2732     449608
            police_per_100k_lag RTC_LAW_YEAR RTC_LAW TIME_0 TIME_INF
Alaska-1980            194.7218         1995   FALSE   1980     2010
Alaska-1981            200.2299         1995   FALSE   1980     2010
Alaska-1982            191.0553         1995   FALSE   1980     2010
            Viol_crime_rate_1k Viol_crime_rate_1k_log Population_log
Alaska-1980           4.742018               1.556463       12.91085
Alaska-1981           6.061851               1.802015       12.94448
Alaska-1982           6.076404               1.804413       13.01613

Indeed we have now created a pdata.frame object and we can see that the row names show the individual states and time period years.

OK, now we are ready to run our panel linear model on our panel data frame.

To do so we will use the plm() function and we will specify the formula for our model, where the dependent variable Viol_crime_rate_1k_log will be on the left of our ~ sign and all of the independent variables will be listed on the right with + signs in between each.

As discussed above, there are different types of panel data analysis, and which type of model is fit is controlled by the effect and model arguments to plm(). So we need to specify what type of effect we would like to model and what type of model we would like to use.

There are three main options for the effect argument: 1) individual - model for the effect of individual identity 2) time - model for the effect of time 3) twoways - meaning modeling for the effect of both individual identity and time

There are four main options for the model argument:
1) pooling - standard pooled ordinary least squares regression model
2) within - fixed effects model (variation between individuals is ignored, model compares individuals to themselves at different periods of time)
3) between - fixed effects model (variation within individuals from one time point to another is ignored, model compares different individuals at each point of time)
4) random - random effects (each state has a different intercept but force it to follow a normal distribution - requires more assumptions)

Typically it is best to think about what you are trying to evaluate with your data in trying to choose how to model your data. However, there are also some tests that can help to assess this which we will briefly cover.

We are interested in how violence in each state varied over time, thus we are interested in within STATEvariation, so we will perform our PLM analysis with the model = within argument to perform this particular type of fixed effects model.

We also speculate that there is an effect of individual STATE identity and time on violent crime rate. In other words, we expect some states to have high rates of crime, and others to have low rates of crime. We also expect crime to change over time. This means we want to use the effect = "twoways" argument to plm().

Here is how we would implement this for the Donohue data:

DONOHUE_OUTPUT <- plm(Viol_crime_rate_1k_log ~
                      RTC_LAW +
                      White_Male_15_to_19_years +
                      White_Male_20_to_39_years +
                      Black_Male_15_to_19_years +
                      Black_Male_20_to_39_years +
                      Other_Male_15_to_19_years +
                      Other_Male_20_to_39_years +
                      Unemployment_rate +
                      Poverty_rate +
                      Population_log +
                      police_per_100k_lag,
                      effect = "twoways",
                      model = "within",
                      data = d_panel_DONOHUE
                      )

To see the results we can use the base summary() function. We can view this output in tidy format using the tidy() function of the broom package.

We will add an analysis variable as a label for plots.

summary(DONOHUE_OUTPUT)

Twoways effects Within Model

Call:
plm(formula = Viol_crime_rate_1k_log ~ RTC_LAW + White_Male_15_to_19_years + 
    White_Male_20_to_39_years + Black_Male_15_to_19_years + Black_Male_20_to_39_years + 
    Other_Male_15_to_19_years + Other_Male_20_to_39_years + Unemployment_rate + 
    Poverty_rate + Population_log + police_per_100k_lag, data = d_panel_DONOHUE, 
    effect = "twoways", model = "within")

Balanced Panel: n = 45, T = 31, N = 1395

Residuals:
       Min.     1st Qu.      Median     3rd Qu.        Max. 
-0.57957437 -0.08942194 -0.00090654  0.08673054  1.11216999 

Coefficients:
                             Estimate  Std. Error t-value  Pr(>|t|)    
RTC_LAWTRUE                0.01796779  0.01663911  1.0799 0.2804066    
White_Male_15_to_19_years -0.00091825  0.02724210 -0.0337 0.9731160    
White_Male_20_to_39_years  0.03466473  0.00972839  3.5633 0.0003794 ***
Black_Male_15_to_19_years -0.05673593  0.05746052 -0.9874 0.3236341    
Black_Male_20_to_39_years  0.12605439  0.01931450  6.5264 9.607e-11 ***
Other_Male_15_to_19_years  0.69201638  0.11322394  6.1119 1.297e-09 ***
Other_Male_20_to_39_years -0.30276797  0.03811855 -7.9428 4.226e-15 ***
Unemployment_rate         -0.01685806  0.00489952 -3.4408 0.0005984 ***
Poverty_rate              -0.00780235  0.00295720 -2.6384 0.0084280 ** 
Population_log            -0.17991653  0.06041773 -2.9779 0.0029559 ** 
police_per_100k_lag        0.00060391  0.00013689  4.4115 1.111e-05 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Total Sum of Squares:    43.211
Residual Sum of Squares: 36.716
R-Squared:      0.15031
Adj. R-Squared: 0.095138
F-statistic: 21.0514 on 11 and 1309 DF, p-value: < 2.22e-16

DONOHUE_OUTPUT_TIDY <- tidy(DONOHUE_OUTPUT, conf.int = 0.95)

DONOHUE_OUTPUT_TIDY

# A tibble: 11 × 7
   term                      estimate std.e…¹ stati…²  p.value conf.low conf.h…³
   <chr>                        <dbl>   <dbl>   <dbl>    <dbl>    <dbl>    <dbl>
 1 RTC_LAWTRUE                1.80e-2 1.66e-2  1.08   2.80e- 1 -1.46e-2  5.06e-2
 2 White_Male_15_to_19_years -9.18e-4 2.72e-2 -0.0337 9.73e- 1 -5.43e-2  5.25e-2
 3 White_Male_20_to_39_years  3.47e-2 9.73e-3  3.56   3.79e- 4  1.56e-2  5.37e-2
 4 Black_Male_15_to_19_years -5.67e-2 5.75e-2 -0.987  3.24e- 1 -1.69e-1  5.59e-2
 5 Black_Male_20_to_39_years  1.26e-1 1.93e-2  6.53   9.61e-11  8.82e-2  1.64e-1
 6 Other_Male_15_to_19_years  6.92e-1 1.13e-1  6.11   1.30e- 9  4.70e-1  9.14e-1
 7 Other_Male_20_to_39_years -3.03e-1 3.81e-2 -7.94   4.23e-15 -3.77e-1 -2.28e-1
 8 Unemployment_rate         -1.69e-2 4.90e-3 -3.44   5.98e- 4 -2.65e-2 -7.26e-3
 9 Poverty_rate              -7.80e-3 2.96e-3 -2.64   8.43e- 3 -1.36e-2 -2.01e-3
10 Population_log            -1.80e-1 6.04e-2 -2.98   2.96e- 3 -2.98e-1 -6.15e-2
11 police_per_100k_lag        6.04e-4 1.37e-4  4.41   1.11e- 5  3.36e-4  8.72e-4
# … with abbreviated variable names ¹std.error, ²statistic, ³conf.high

DONOHUE_OUTPUT_TIDY$Analysis <- "Analysis 1"

As discussed above, we are choosing to use a fixed effects model because we believe it is most appropriate for our data. However, there are statistical tests implemented in the plm package that would allow us to test whether one of the other models would be as appropriate for our data. For example, we could perform a test to determine if we could have simply used a pooled model, i.e., the test evaluates whether the coefficients (including the intercepts) are equal across individuals. This test is implemented in the pooltest() function of the plm package and performs a comparison between the pooled model to the fixed effect within model. To test if using a random effect model would be more appropriate compared to the fixed effect model, one could use the Hausman test (also called the Durbin-Wu-Hausman test). This test is implemented using the phtest() function of the plm package.

Finally, we could also perform a test to evaluate if there is indeed an individual (state) effect and a time effect in our model using the plmtest() function of the plm package.

For more information on these tests and this package, see here and here.

Mustard and Lott

OK, now we will do the same for the Mustard and Lott analysis. In this case we would have a very large formula to write. So instead, we can use the as.formula() function of the stats package and the base paste() function to combine all of our explanatory variables into one formula without making a mistake. First we will create an object where we select only the explanatory variables.

LOTT_variables <- LOTT_DF %>%
  dplyr::select(
    RTC_LAW,
    contains(c("White", "Black", "Other")),
    Unemployment_rate,
    Poverty_rate,
    Population_log,
    police_per_100k_lag
  ) %>%
  colnames()


LOTT_fmla <- as.formula(paste(
  "Viol_crime_rate_1k_log ~",
  paste(LOTT_variables, collapse = " + ")
))

LOTT_fmla

Viol_crime_rate_1k_log ~ RTC_LAW + White_Female_10_to_19_years + 
    White_Female_20_to_29_years + White_Female_30_to_39_years + 
    White_Female_40_to_49_years + White_Female_50_to_64_years + 
    White_Female_65_years_and_over + White_Male_10_to_19_years + 
    White_Male_20_to_29_years + White_Male_30_to_39_years + White_Male_40_to_49_years + 
    White_Male_50_to_64_years + White_Male_65_years_and_over + 
    Black_Female_10_to_19_years + Black_Female_20_to_29_years + 
    Black_Female_30_to_39_years + Black_Female_40_to_49_years + 
    Black_Female_50_to_64_years + Black_Female_65_years_and_over + 
    Black_Male_10_to_19_years + Black_Male_20_to_29_years + Black_Male_30_to_39_years + 
    Black_Male_40_to_49_years + Black_Male_50_to_64_years + Black_Male_65_years_and_over + 
    Other_Female_10_to_19_years + Other_Female_20_to_29_years + 
    Other_Female_30_to_39_years + Other_Female_40_to_49_years + 
    Other_Female_50_to_64_years + Other_Female_65_years_and_over + 
    Other_Male_10_to_19_years + Other_Male_20_to_29_years + Other_Male_30_to_39_years + 
    Other_Male_40_to_49_years + Other_Male_50_to_64_years + Other_Male_65_years_and_over + 
    Unemployment_rate + Poverty_rate + Population_log + police_per_100k_lag

That is quite the formula!

OK, now again we will make a panel data frame and we will fit a fixed effect two-way model for time and individuals (STATE) with this data as well.

d_panel_LOTT <- pdata.frame(LOTT_DF, index = c("STATE", "YEAR"))

LOTT_OUTPUT <- plm(LOTT_fmla,
  model = "within",
  effect = "twoways",
  data = d_panel_LOTT
)

summary(LOTT_OUTPUT)

Twoways effects Within Model

Call:
plm(formula = LOTT_fmla, data = d_panel_LOTT, effect = "twoways", 
    model = "within")

Balanced Panel: n = 45, T = 31, N = 1395

Residuals:
      Min.    1st Qu.     Median    3rd Qu.       Max. 
-0.5448906 -0.0780395  0.0026738  0.0788052  0.5847263 

Coefficients:
                                  Estimate  Std. Error  t-value  Pr(>|t|)    
RTC_LAWTRUE                    -0.04687169  0.01641851  -2.8548 0.0043758 ** 
White_Female_10_to_19_years     0.62441376  0.15103427   4.1343 3.793e-05 ***
White_Female_20_to_29_years    -0.05942541  0.06332108  -0.9385 0.3481763    
White_Female_30_to_39_years     0.16028113  0.08045953   1.9921 0.0465755 *  
White_Female_40_to_49_years     0.10087510  0.08170707   1.2346 0.2172082    
White_Female_50_to_64_years    -0.37624966  0.06303172  -5.9692 3.083e-09 ***
White_Female_65_years_and_over  0.20636690  0.04742430   4.3515 1.460e-05 ***
White_Male_10_to_19_years      -0.59141591  0.14436974  -4.0965 4.457e-05 ***
White_Male_20_to_29_years       0.08717546  0.05862342   1.4870 0.1372503    
White_Male_30_to_39_years      -0.12514225  0.08588569  -1.4571 0.1453400    
White_Male_40_to_49_years      -0.21812366  0.07293615  -2.9906 0.0028375 ** 
White_Male_50_to_64_years       0.37845575  0.07314122   5.1743 2.653e-07 ***
White_Male_65_years_and_over   -0.20915907  0.06659815  -3.1406 0.0017246 ** 
Black_Female_10_to_19_years    -1.03146594  0.43610403  -2.3652 0.0181697 *  
Black_Female_20_to_29_years    -0.02721685  0.17462559  -0.1559 0.8761693    
Black_Female_30_to_39_years    -0.03246043  0.20498789  -0.1584 0.8742037    
Black_Female_40_to_49_years     0.43820099  0.23524130   1.8628 0.0627234 .  
Black_Female_50_to_64_years     0.04906111  0.21393128   0.2293 0.8186482    
Black_Female_65_years_and_over  0.07226074  0.24373031   0.2965 0.7669130    
Black_Male_10_to_19_years       1.22536162  0.44559642   2.7499 0.0060447 ** 
Black_Male_20_to_29_years      -0.06587312  0.18392655  -0.3581 0.7202909    
Black_Male_30_to_39_years       0.24720746  0.23673862   1.0442 0.2965804    
Black_Male_40_to_49_years      -0.66869983  0.27173041  -2.4609 0.0139904 *  
Black_Male_50_to_64_years      -0.16737616  0.23977741  -0.6980 0.4852740    
Black_Male_65_years_and_over   -0.58743446  0.34691532  -1.6933 0.0906404 .  
Other_Female_10_to_19_years     0.70957924  0.49539878   1.4323 0.1522910    
Other_Female_20_to_29_years    -1.16489945  0.26997487  -4.3148 1.720e-05 ***
Other_Female_30_to_39_years    -3.40258912  0.35368437  -9.6204 < 2.2e-16 ***
Other_Female_40_to_49_years     1.34563633  0.42503994   3.1659 0.0015825 ** 
Other_Female_50_to_64_years     2.93990932  0.33830653   8.6901 < 2.2e-16 ***
Other_Female_65_years_and_over  2.36026239  0.20422580  11.5571 < 2.2e-16 ***
Other_Male_10_to_19_years       0.07481449  0.47835310   0.1564 0.8757423    
Other_Male_20_to_29_years       1.62895925  0.25740603   6.3284 3.420e-10 ***
Other_Male_30_to_39_years       3.17421278  0.41184489   7.7073 2.566e-14 ***
Other_Male_40_to_49_years      -1.58494177  0.44840281  -3.5346 0.0004229 ***
Other_Male_50_to_64_years      -3.91523867  0.37399898 -10.4686 < 2.2e-16 ***
Other_Male_65_years_and_over   -4.16596244  0.36860536 -11.3020 < 2.2e-16 ***
Unemployment_rate              -0.00545734  0.00436374  -1.2506 0.2113054    
Poverty_rate                   -0.00572362  0.00253162  -2.2609 0.0239357 *  
Population_log                 -0.21716335  0.08452664  -2.5692 0.0103068 *  
police_per_100k_lag             0.00069547  0.00013331   5.2171 2.118e-07 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Total Sum of Squares:    43.211
Residual Sum of Squares: 23.647
R-Squared:      0.45275
Adj. R-Squared: 0.40355
F-statistic: 25.8088 on 41 and 1279 DF, p-value: < 2.22e-16

LOTT_OUTPUT_TIDY <- tidy(LOTT_OUTPUT, conf.int = 0.95)

LOTT_OUTPUT_TIDY$Analysis <- "Analysis 2"

RTC coefficient comparison

Now let’s make a plot to compare the coefficient estimate for the Right-to-carry law adoption variable in each model.

First we will combine model fit information for this coefficient for each model.

comparing_analyses <- DONOHUE_OUTPUT_TIDY %>%
  bind_rows(LOTT_OUTPUT_TIDY) %>%
  filter(term == "RTC_LAWTRUE")

comparing_analyses

# A tibble: 2 × 8
  term        estimate std.error statistic p.value conf.low conf.high Analysis  
  <chr>          <dbl>     <dbl>     <dbl>   <dbl>    <dbl>     <dbl> <chr>     
1 RTC_LAWTRUE   0.0180    0.0166      1.08 0.280    -0.0146    0.0506 Analysis 1
2 RTC_LAWTRUE  -0.0469    0.0164     -2.85 0.00438  -0.0791   -0.0147 Analysis 2

We can see that for the first analysis (similar to the Donohue et al. study) the coefficient estimate for the presence of a permissive right-to-carry law is positive, while for the second analysis (similar to the Mustard and Lott study) the coefficient estimate is negative. Thus in the first analysis we could conclude that the effect of adopting permissive right-to-carry laws may be associated with increases in violent crime (although this was not a significant result (in contrast with the real Donohue et al. study )); while in the other analysis we could conclude that the laws may be associated with decreases in violent crime.

Let’s make a plot of this finding. We will show error bars for the coefficient estimates for both analyses using the geom_errorbar() function of the ggplot2 package. This requires specifying the minimum and maximum for our error bar, which in our case will be the low and high values of our confidence intervals for the coefficient estimates. We will also add a horizontal line at y = 0 using the geom_hline() function of the ggplot2 package.

Finally we will add arrows to emphasize the difference in the direction of the findings using the geom_segment() function of the ggplot2 package. Using the arrow() function, we can specify details about the arrow we would like to add.

comparing_analyses_plot <- ggplot(comparing_analyses) +
  geom_point(aes(x = Analysis, y = estimate)) +
  geom_errorbar(aes(x = Analysis, ymin = conf.low, ymax = conf.high), width = 0.25) +
  geom_hline(yintercept = 0, color = "red") +
  scale_y_continuous(
    breaks = seq(-0.2, 0.2, by = 0.05),
    labels = seq(-0.2, 0.2, by = 0.05),
    limits = c(-0.2, 0.2)
  ) +
  geom_segment(aes(x = 1, y = 0.125, xend = 1, yend = 0.175),
    arrow = arrow(angle = 45, ends = "last", type = "open"),
    size = 2,
    color = "green",
    lineend = "butt",
    linejoin = "mitre"
  ) +
  geom_segment(aes(x = 2, y = -0.125, xend = 2, yend = -0.175),
    arrow = arrow(angle = 45, ends = "last", type = "open"),
    size = 2,
    color = "red",
    lineend = "butt",
    linejoin = "mitre"
  ) +
  theme_minimal() +
  theme(
    axis.title.x = element_blank(),
    axis.text = element_text(size = 8, color = "black")
  ) +
  labs(
    title = "Effect estimate on ln(violent crimes per 100,000 people)",
    y = "  Effect estimate (95% CI)"
  )

comparing_analyses_plot

We can see that the confidence interval from analysis 1 is mostly covering positive values, while the entire confidence interval is negative for analysis 2.

Multicollinearity analysis

How did the above happen?

The analysis data frames are very similar yet yielded very different results.

Recall that the only difference between the two models is the number of demographic variables included as covariates. The number of rows or observations is the same, as are the outcome and the other covariates included in the model. We can use the all_equal() function of the dplyr package to compare the number of columns of our Donohue-like data and our Mustard and Lott-like data.

all_equal(
  target = DONOHUE_DF,
  current = LOTT_DF,
  ignore_col_order = TRUE,
  ignore_row_order = TRUE
)

Different number of columns: 20 vs 50.

Using the base dim() function we can also look at the number of rows for each and see that the number of observations is the same for both datasets.

dim(DONOHUE_DF)[1]

[1] 1395

dim(LOTT_DF)[1]

[1] 1395

The only difference between the two data frames rests in how the demographic variables were parameterized.

DONOHUE_DF %>%
  dplyr::select(contains("years")) %>%
  colnames()

[1] "Black_Male_15_to_19_years" "Black_Male_20_to_39_years"
[3] "Other_Male_15_to_19_years" "Other_Male_20_to_39_years"
[5] "White_Male_15_to_19_years" "White_Male_20_to_39_years"

LOTT_DF %>%
  dplyr::select(contains("years")) %>%
  colnames()

 [1] "Black_Female_10_to_19_years"    "Black_Female_20_to_29_years"   
 [3] "Black_Female_30_to_39_years"    "Black_Female_40_to_49_years"   
 [5] "Black_Female_50_to_64_years"    "Black_Female_65_years_and_over"
 [7] "Black_Male_10_to_19_years"      "Black_Male_20_to_29_years"     
 [9] "Black_Male_30_to_39_years"      "Black_Male_40_to_49_years"     
[11] "Black_Male_50_to_64_years"      "Black_Male_65_years_and_over"  
[13] "Other_Female_10_to_19_years"    "Other_Female_20_to_29_years"   
[15] "Other_Female_30_to_39_years"    "Other_Female_40_to_49_years"   
[17] "Other_Female_50_to_64_years"    "Other_Female_65_years_and_over"
[19] "Other_Male_10_to_19_years"      "Other_Male_20_to_29_years"     
[21] "Other_Male_30_to_39_years"      "Other_Male_40_to_49_years"     
[23] "Other_Male_50_to_64_years"      "Other_Male_65_years_and_over"  
[25] "White_Female_10_to_19_years"    "White_Female_20_to_29_years"   
[27] "White_Female_30_to_39_years"    "White_Female_40_to_49_years"   
[29] "White_Female_50_to_64_years"    "White_Female_65_years_and_over"
[31] "White_Male_10_to_19_years"      "White_Male_20_to_29_years"     
[33] "White_Male_30_to_39_years"      "White_Male_40_to_49_years"     
[35] "White_Male_50_to_64_years"      "White_Male_65_years_and_over"

Clearly, this had an effect on the results of the analysis.

Let’s explore how this occurred.

When seemingly independent variables are highly related to one another, the relationships estimated in an analysis may be distorted.

In regression analysis, this distortion is often a by-product of a violation of the independence assumption. This distortion, if large enough, can impact statistical inference.

The phenomenon called multicollinearity occurs when independent variables are highly related to one another.

There are several ways we can diagnose multicollinearity.

Correlation

One way we can evaluate the relationships between variables is by examining the correlation between variable pairs.

It is important to note that multicollinearity and correlation are not the same concept. Correlation can be thought of as the strength of a linear relationship between variables in general. On the other hand, we usually use the term collinearity in the context of a multiple regression model, where two independent variables are collinear if they have a linear relationship or association. Multicollinearity can be thought of as collinearity among multiple (3+) regressors (independent variables) in a regression analysis, which can occur when regressors are highly correlated.

According to Wikipedia:

multicollinearity (also collinearity) is a phenomenon in which one predictor variable in a multiple regression model can be linearly predicted from the others with a substantial degree of accuracy.

Thus collinearity describes linear prediction or association between variables. Often those variables will be highly correlated.

The issue with this in linear regression, is that linear regression aims to determine how a one unit change in a regressor influences a one unit change in the dependent variable. In fact, this is what the coefficient estimates aim to tell us for each regressor.

However, if our regressors are also linearly related or correlated, then it becomes difficult to determine the effect of each regressor on the dependent variable, variable independent of the other regressors, and multicollinearity can cause instability in our coefficient estimates, making them unreliable. In the presence of multicollinearity, coefficient estimates may be inflated, deflated, or their signs may change.

For example, say both waist and hip circumference are included as predictors of BMI in a multiple regression model. As in any multiple regression model, the coefficient of waist circumference represents the average change in BMI for a one unit change in waist circumference, holding the other predictors in the model constant. Since it is not very meaningful to consider variation in waist circumference while holding hip circumference constant (as waist and hip circumference are indeed related to one another), our estimate of this coefficient may be uncertain.

If you want to read further on this topic, here and here are a couple of interesting discussions.

In the next sections, we will describe ways to detect multicollinearity in our covariates both using visual displays of data and using computational techniques.

Scatter plots

One of the ways to diagnose multicollinearity in a regression model is to first see if there are regressors that are highly correlated. If so, this suggests that we should investigate further to see if these variables are in fact linearly predicting one another.

One way to look at correlation across pairs of variables is to use the ggpairs() function of the GGally package.

colnames(DONOHUE_DF)

 [1] "YEAR"                      "STATE"                    
 [3] "Black_Male_15_to_19_years" "Black_Male_20_to_39_years"
 [5] "Other_Male_15_to_19_years" "Other_Male_20_to_39_years"
 [7] "White_Male_15_to_19_years" "White_Male_20_to_39_years"
 [9] "Unemployment_rate"         "Poverty_rate"             
[11] "Viol_crime_count"          "Population"               
[13] "police_per_100k_lag"       "RTC_LAW_YEAR"             
[15] "RTC_LAW"                   "TIME_0"                   
[17] "TIME_INF"                  "Viol_crime_rate_1k"       
[19] "Viol_crime_rate_1k_log"    "Population_log"

DONOHUE_DF %>%
  dplyr::select(
    RTC_LAW,
    Viol_crime_rate_1k_log,
    Unemployment_rate,
    Poverty_rate,
    Population_log
  ) %>%
  ggpairs(.,
    columns = c(2:5),
    lower = list(continuous = wrap("smooth_loess",
      color = "red",
      alpha = 0.5,
      size = 0.1
    ))
  )

We can see that for the non-demographic variables, there is very little correlation between the pairs of variables. Only the unemployment rate and the poverty rate show relatively strong correlation, as one might expect.

Heatmaps

Another way to look at correlation if we have many variables is to show the strength of correlation between pairs of variables using a heatmap, where the intensity of the color indicates the strength of the correlation between two variables.

Let’s do this now for the demographic variables for each analysis.

The ggcorrplot() function of the ggpcorrplot package is one way to create such a heatmap.

As input to the plotting function, we first need to calculate the correlation values, which we will do using the cor() function of the stats package.

To label our legend with the Greek letter $\rho$, we will use the base expression() function, which will convert the written form of "rho" to the Greek letter.

cor_DONOHUE_dem <- cor(DONOHUE_DF %>% dplyr::select(contains("_years")))

corr_mat_DONOHUE <- ggcorrplot(cor_DONOHUE_dem,
  tl.cex = 6,
  hc.order = TRUE,
  colors = c(
    "red",
    "white",
    "red"
  ),
  outline.color = "transparent",
  title = "Correlation Matrix, Analysis 1",
  legend.title = expression(rho)
)


corr_mat_DONOHUE

cor_LOTT_dem <- cor(LOTT_DF %>% dplyr::select(contains("_years")))

corr_mat_LOTT <- ggcorrplot(cor_LOTT_dem,
  tl.cex = 6,
  hc.order = TRUE,
  colors = c(
    "red",
    "white",
    "red"
  ),
  outline.color = "transparent",
  title = "Correlation Matrix, Analysis 2",
  legend.title = expression(rho)
)

corr_mat_LOTT

We can see that many of the demographic variables are highly correlated with one another, either positively or negatively. In this case, the sign does not matter, in terms of the effect the collinearity could have on our modeling results.

The presence of correlation between variables suggests that we might have multicollinearity. However it does not necessarily mean that we do. So how can we assess this?

Coefficient estimate instability

One way to look at the possible influence of multicollinearity is to look at the stability of the coefficient estimates under perturbations of the data.

We will focus on the RTC_LAW variable coefficient estimate, as this is of particular interest in our case.

To do so we will perform a process called resampling. This involves performing multiple iterations of our analysis, but with only a subset or sub-sample of our original data. In our case we will remove one observation and see if that changes our coefficient estimate results.

To do this we will use some functions in the rsample package which is very useful for splitting data in various ways.

We will use the loo_cv() function which stands for leave one out cross validation. This will allow us to split our data into every possible subset where a unique observation is left out of the data.

This function will however only prepare the data to be split.

To get the remaining data after the removal of the observation that is left out we will use a function called training(). These function names arise from the fact that these functions are often used for in machine learning applications where the data is split between a larger training set and a smaller testing set. Thus we want the larger $n-1$ subset, as opposed to the single value that is removed, (which we could get with the testing() function).

Click here to see an example of how this works.

First we will make a toy dataset that is very simple called test using the tibble() function of the tibble package:

test <- tibble::tibble(x = c(1, 2, 3))
test

# A tibble: 3 × 1
      x
  <dbl>
1     1
2     2
3     3

Now we will use the loo_cv() to create leave one out splits:

test_samples <- test %>% rsample::loo_cv()
test_samples

# Leave-one-out cross-validation 
# A tibble: 3 × 2
  splits        id       
  <list>        <chr>    
1 <split [2/1]> Resample1
2 <split [2/1]> Resample2
3 <split [2/1]> Resample3

We can take a look at a single split of the data using the pull() function:

pull(test_samples, splits)

[[1]]
<Analysis/Assess/Total>
<2/1/3>

[[2]]
<Analysis/Assess/Total>
<2/1/3>

[[3]]
<Analysis/Assess/Total>
<2/1/3>

Here you can see that 2 values are intended for the training set (also called Analysis set), 1 value is intended for the testing set (also called Assessment set), and 3 values were present initially.

Now we will use the training() function to get the data without the observation that is set aside. Here is the data for the first subset:

rsample::training(pull(test_samples, splits)[[1]])

# A tibble: 2 × 1
      x
  <dbl>
1     2
2     3

Now we will use the map() function of purrr to get all possible training subset of the data.

test_subsets <- map(pull(test_samples, splits), training)
test_subsets

[[1]]
# A tibble: 2 × 1
      x
  <dbl>
1     2
2     3

[[2]]
# A tibble: 2 × 1
      x
  <dbl>
1     1
2     3

[[3]]
# A tibble: 2 × 1
      x
  <dbl>
1     1
2     2

We can see that there are 3 possible subsets that leave one value out. All 3 possible subsets are created using this method. This method will always create the same number of subsets as there are unique values or rows in the data.

Now we will use this method with the data from our Donohue-like analysis, and since this data has 1395 rows, 1395 subsets will be created that leave out one row. The idea is to fit our panel regression model on each subset of the data, and then examine how the coefficient estimates from each of these model fits vary as the sample changes slightly. With collinear predictors, we expect that our coefficient estimates may be unstable and subject to change under even small perturbations of the data.

First we will create the splits using the loo_cv() function. This will ultimately make all unique possible subsets leaving one sample out - however the choice of which sample is left for which iteration is a random process (maybe the 12th is left out first then the 3rd etc.). Thus, we use the set.seed() function to ensure reproducibility of the results when we take a look at the first subset, so that it will be the same for everyone who looks at this case study. You can change the seed to play around with this.

set.seed(124)
DONOHUE_splits <- d_panel_DONOHUE %>% loo_cv()
DONOHUE_splits

# Leave-one-out cross-validation 
# A tibble: 1,395 × 2
   splits           id        
   <list>           <chr>     
 1 <split [1394/1]> Resample1 
 2 <split [1394/1]> Resample2 
 3 <split [1394/1]> Resample3 
 4 <split [1394/1]> Resample4 
 5 <split [1394/1]> Resample5 
 6 <split [1394/1]> Resample6 
 7 <split [1394/1]> Resample7 
 8 <split [1394/1]> Resample8 
 9 <split [1394/1]> Resample9 
10 <split [1394/1]> Resample10
# … with 1,385 more rows

Now we will use the training() function to select the remaining data without the value that was removed for each split:

# To get all the data subsets
DONOHUE_subsets <- map(pull(DONOHUE_splits, splits), training)

glimpse(DONOHUE_subsets[[1]])

Rows: 1,394
Columns: 20
$ YEAR                      <fct> 1980, 1981, 1982, 1983, 1984, 1985, 1986, 19…
$ STATE                     <fct> Alaska, Alaska, Alaska, Alaska, Alaska, Alas…
$ Black_Male_15_to_19_years <pseries> 0.1670456, 0.1732299, 0.1737069, 0.17095…
$ Black_Male_20_to_39_years <pseries> 0.9933775, 1.0028219, 1.0204445, 1.03127…
$ Other_Male_15_to_19_years <pseries> 1.1297816, 1.1244412, 1.0698208, 0.98828…
$ Other_Male_20_to_39_years <pseries> 2.963329, 2.974775, 3.015071, 3.008048, …
$ White_Male_15_to_19_years <pseries> 3.627805, 3.558261, 3.391844, 3.222002, …
$ White_Male_20_to_39_years <pseries> 18.28852, 18.12821, 18.10666, 17.90600, …
$ Unemployment_rate         <pseries> 9.6, 9.4, 9.9, 9.9, 9.8, 9.7, 10.9, 10.3…
$ Poverty_rate              <pseries> 9.6, 9.0, 10.6, 12.6, 9.6, 8.7, 11.4, 12…
$ Viol_crime_count          <pseries> 1919, 2537, 2732, 2940, 3108, 3031, 3046…
$ Population                <pseries> 404680, 418519, 449608, 488423, 513697, …
$ police_per_100k_lag       <pseries> 194.7218, 200.2299, 191.0553, 364.2335, …
$ RTC_LAW_YEAR              <pseries> 1995, 1995, 1995, 1995, 1995, 1995, 1995…
$ RTC_LAW                   <pseries> FALSE, FALSE, FALSE, FALSE, FALSE, FALSE…
$ TIME_0                    <pseries> 1980, 1980, 1980, 1980, 1980, 1980, 1980…
$ TIME_INF                  <pseries> 2010, 2010, 2010, 2010, 2010, 2010, 2010…
$ Viol_crime_rate_1k        <pseries> 4.742018, 6.061851, 6.076404, 6.019373, …
$ Viol_crime_rate_1k_log    <pseries> 1.556463, 1.802015, 1.804413, 1.794983, …
$ Population_log            <pseries> 12.91085, 12.94448, 13.01613, 13.09894, …

length(DONOHUE_subsets)

[1] 1395

As expected the first subset has 1394 rows and there are 1395 subsets.

Let’s see what observation was left out in the first subset:

d_panel_DONOHUE %>%
  filter(!rownames(d_panel_DONOHUE) %in% rownames(DONOHUE_subsets[[1]]))

           YEAR STATE Black_Male_15_to_19_years Black_Male_20_to_39_years
Texas-1988 1988 Texas                 0.5599219                  2.091018
           Other_Male_15_to_19_years Other_Male_20_to_39_years
Texas-1988                0.09824717                 0.4413413
           White_Male_15_to_19_years White_Male_20_to_39_years
Texas-1988                  3.447339                  14.79001
           Unemployment_rate Poverty_rate Viol_crime_count Population
Texas-1988               7.3           18           109499   16667146
           police_per_100k_lag RTC_LAW_YEAR RTC_LAW TIME_0 TIME_INF
Texas-1988            298.2274         1996   FALSE   1980     2010
           Viol_crime_rate_1k Viol_crime_rate_1k_log Population_log
Texas-1988           6.569751               1.882476       16.62895

# Another way to check is to use:
DONOHUE_removed <- map(pull(DONOHUE_splits, splits), testing)

DONOHUE_removed[[1]]

           YEAR STATE Black_Male_15_to_19_years Black_Male_20_to_39_years
Texas-1988 1988 Texas                 0.5599219                  2.091018
           Other_Male_15_to_19_years Other_Male_20_to_39_years
Texas-1988                0.09824717                 0.4413413
           White_Male_15_to_19_years White_Male_20_to_39_years
Texas-1988                  3.447339                  14.79001
           Unemployment_rate Poverty_rate Viol_crime_count Population
Texas-1988               7.3           18           109499   16667146
           police_per_100k_lag RTC_LAW_YEAR RTC_LAW TIME_0 TIME_INF
Texas-1988            298.2274         1996   FALSE   1980     2010
           Viol_crime_rate_1k Viol_crime_rate_1k_log Population_log
Texas-1988           6.569751               1.882476       16.62895

It looks like the Texas data from 1988 was removed from the first split. Again, if you try different seeds you will see a different sample removed from the first split.

OK, so now let’s fit our panel regression on the first subset of data like we did previously. Note that this causes our data to be an unbalanced panel. This does not require any adjustment to the code to model the data, but you will notice that the output will now say “unbalanced”.

subset_1_result <- plm(Viol_crime_rate_1k_log ~
RTC_LAW +
  White_Male_15_to_19_years +
  White_Male_20_to_39_years +
  Black_Male_15_to_19_years +
  Black_Male_20_to_39_years +
  Other_Male_15_to_19_years +
  Other_Male_20_to_39_years +
  Unemployment_rate +
  Poverty_rate +
  Population_log +
  police_per_100k_lag,
data = DONOHUE_subsets[[1]],
index = c("STATE", "YEAR"),
model = "within",
effect = "twoways"
)
summary(subset_1_result)

Twoways effects Within Model

Call:
plm(formula = Viol_crime_rate_1k_log ~ RTC_LAW + White_Male_15_to_19_years + 
    White_Male_20_to_39_years + Black_Male_15_to_19_years + Black_Male_20_to_39_years + 
    Other_Male_15_to_19_years + Other_Male_20_to_39_years + Unemployment_rate + 
    Poverty_rate + Population_log + police_per_100k_lag, data = DONOHUE_subsets[[1]], 
    effect = "twoways", model = "within", index = c("STATE", 
        "YEAR"))

Unbalanced Panel: n = 45, T = 30-31, N = 1394

Residuals:
      Min.    1st Qu.     Median    3rd Qu.       Max. 
-0.5795659 -0.0893732 -0.0014097  0.0865870  1.1118927 

Coefficients:
                             Estimate  Std. Error t-value  Pr(>|t|)    
RTC_LAWTRUE                0.01812436  0.01664364  1.0890 0.2763694    
White_Male_15_to_19_years -0.00103284  0.02724763 -0.0379 0.9697687    
White_Male_20_to_39_years  0.03462260  0.00973037  3.5582 0.0003868 ***
Black_Male_15_to_19_years -0.05699742  0.05747236 -0.9917 0.3215097    
Black_Male_20_to_39_years  0.12591876  0.01931901  6.5179 1.016e-10 ***
Other_Male_15_to_19_years  0.69114956  0.11325143  6.1028 1.371e-09 ***
Other_Male_20_to_39_years -0.30242747  0.03812860 -7.9318 4.603e-15 ***
Unemployment_rate         -0.01698131  0.00490345 -3.4631 0.0005512 ***
Poverty_rate              -0.00782727  0.00295795 -2.6462 0.0082384 ** 
Population_log            -0.17899715  0.06044257 -2.9614 0.0031173 ** 
police_per_100k_lag        0.00060326  0.00013692  4.4058 1.140e-05 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Total Sum of Squares:    43.198
Residual Sum of Squares: 36.701
R-Squared:      0.15039
Adj. R-Squared: 0.095183
F-statistic: 21.0489 on 11 and 1308 DF, p-value: < 2.22e-16

Indeed, we can see that now we have an unbalanced panel with N = 1394 observations instead of 1395, as expected.

Now that we have our subsets, we want to write a function to fit a panel regression using plm()on each subset. See this case study for more information on writing functions.

fit_nls_on_bootstrap_DONOHUE <- function(subset) {
  plm(Viol_crime_rate_1k_log ~
  RTC_LAW +
    White_Male_15_to_19_years +
    White_Male_20_to_39_years +
    Black_Male_15_to_19_years +
    Black_Male_20_to_39_years +
    Other_Male_15_to_19_years +
    Other_Male_20_to_39_years +
    Unemployment_rate +
    Poverty_rate +
    Population_log +
    police_per_100k_lag,
  data = data.frame(subset),
  index = c("STATE", "YEAR"),
  model = "within",
  effect = "twoways"
  )
}

Now we can apply this function to each of our subsets simultaneously using the map() function of the purrr package.

subsets_models_DONOHUE <- map(DONOHUE_subsets, fit_nls_on_bootstrap_DONOHUE)

subsets_models_DONOHUE <- subsets_models_DONOHUE %>%
  map(tidy)

We will save this data in our data directory within a subdirectory called wrangled like so:

save(subsets_models_DONOHUE,
  file = here::here("data", "wrangled", "DONOHUE_simulations.rda")
)

If you have trouble with the previous step you can load the data like so:

load(here::here("data", "wrangled", "DONOHUE_simulations.rda"))

Great! Now we want to do the same thing for the Mustard and Lott data.

set.seed(124)
LOTT_splits <- d_panel_LOTT %>% loo_cv()

# To get all the data subsets:
LOTT_subsets <- map(pull(LOTT_splits, splits), training)

We need to create a different function to fit the data to account for the larger number of demographic variables. We will use the formula that we made previously.

fit_nls_on_bootstrap_LOTT <- function(split) {
  plm(LOTT_fmla,
    data = data.frame(split),
    index = c("STATE", "YEAR"),
    model = "within",
    effect = "twoways"
  )
}

subsets_models_LOTT <- map(LOTT_subsets, fit_nls_on_bootstrap_LOTT)

subsets_models_LOTT <- subsets_models_LOTT %>%
  map(tidy)

Again we will save this in the wrangled subdirectory of our data directory:

save(subsets_models_LOTT,
  file = here::here("data", "wrangled", "LOTT_simulations.rda")
)

If you have trouble with the previous steps you can load this data like so:

load(here::here("data", "wrangled", "LOTT_simulations.rda"))

Now we will combine the output so that we can make a plot to visualize the results that we obtained, i.e., to look at how the variation in our coefficient estimates across data subsets varies between the Donohue and the Lott and Mustard models. First let’s name each subset that we created.

names(subsets_models_DONOHUE) <- paste0("DONOHUE_", seq_len(length(subsets_models_DONOHUE)))

names(subsets_models_LOTT) <-
  paste0("LOTT_", 1:length(subsets_models_LOTT))

Now we can combine the tibbles within the list of tibbles for the subsets_models_DONOHUE and subsets_models_LOTT data.

To do this we will use the bind_rows() function of the dplyr package with the .id = "ID" argument, which will create a new variable called ID that will list the name of the tibble the data came from.

Then we will combine the data from both the Donohue and Lott simulations.

simulations_DONOHUE <- subsets_models_DONOHUE %>%
  bind_rows(.id = "ID") %>%
  mutate(Analysis = "Analysis 1")

simulations_LOTT <- subsets_models_LOTT %>%
  bind_rows(.id = "ID") %>%
  mutate(Analysis = "Analysis 2")

simulations <- bind_rows(
  simulations_DONOHUE,
  simulations_LOTT
)

head(simulations)

# A tibble: 6 × 7
  ID        term                      estimate std.er…¹ stati…²  p.value Analy…³
  <chr>     <chr>                        <dbl>    <dbl>   <dbl>    <dbl> <chr>  
1 DONOHUE_1 RTC_LAWTRUE                0.0181   0.0166   1.09   2.76e- 1 Analys…
2 DONOHUE_1 White_Male_15_to_19_years -0.00103  0.0272  -0.0379 9.70e- 1 Analys…
3 DONOHUE_1 White_Male_20_to_39_years  0.0346   0.00973  3.56   3.87e- 4 Analys…
4 DONOHUE_1 Black_Male_15_to_19_years -0.0570   0.0575  -0.992  3.22e- 1 Analys…
5 DONOHUE_1 Black_Male_20_to_39_years  0.126    0.0193   6.52   1.02e-10 Analys…
6 DONOHUE_1 Other_Male_15_to_19_years  0.691    0.113    6.10   1.37e- 9 Analys…
# … with abbreviated variable names ¹std.error, ²statistic, ³Analysis

tail(simulations)

# A tibble: 6 × 7
  ID        term                       estimate std.e…¹ stati…²  p.value Analy…³
  <chr>     <chr>                         <dbl>   <dbl>   <dbl>    <dbl> <chr>  
1 LOTT_1395 Other_Male_50_to_64_years  -3.91e+0 3.75e-1  -10.4  1.66e-24 Analys…
2 LOTT_1395 Other_Male_65_years_and_o… -4.16e+0 3.69e-1  -11.3  3.97e-28 Analys…
3 LOTT_1395 Unemployment_rate          -5.43e-3 4.37e-3   -1.24 2.14e- 1 Analys…
4 LOTT_1395 Poverty_rate               -5.75e-3 2.53e-3   -2.27 2.33e- 2 Analys…
5 LOTT_1395 Population_log             -2.16e-1 8.46e-2   -2.55 1.08e- 2 Analys…
6 LOTT_1395 police_per_100k_lag         6.96e-4 1.33e-4    5.22 2.07e- 7 Analys…
# … with abbreviated variable names ¹std.error, ²statistic, ³Analysis

Now we will make a set of parallel boxplots using the geom_boxplot() function of the coefficient estimates of the RTC_LAWTRUE variable for each simulation.

Since there are many variables in both analyses, we will use the facet_grid() function of the ggplot2 package to allow us to separate the data for each analysis into subplots. The argument scale = "free_x" and drop = TRUE allow us to only include the variables that were present in Analysis 1, as opposed to empty spots for the variables that were in Analysis 2 but not in Analysis 1. The space = "free" argument removes the extra space from the dropped variables.

Question Opportunity

What happens if you don’t use the drop = TRUE argument or the space = "free" argument?

simulation_plot <- simulations %>%
  ggplot(aes(x = term, y = estimate)) +
  geom_boxplot() +
  facet_grid(. ~ Analysis, scale = "free_x", space = "free", drop = TRUE) +
  labs(
    title = "Coefficient estimates",
    subtitle = "Estimates across leave-one-out analyses",
    x = "Term",
    y = "Coefficient",
    caption = "Results from simulations"
  ) +
  theme_linedraw() +
  theme(
    axis.title.x = element_blank(),
    axis.text.x = element_text(angle = 70, hjust = 1),
    strip.text.x = element_text(size = 14, face = "bold")
  )

simulation_plot

Here, we can start to see that there is a bit more variability in the coefficient estimates from the leave-one-out results for Analysis 2.

For our display purposes, we would like to order the covariates so that they are displayed similarly across the two panels. This will allow us to better observe how the coefficients of the same covariate behave in the different analyses. We use the mutate function to convert the term variable to a factor, where we assign the non-demographic variables to be the first levels, with the sorted (in alphabetical order using the base sort() function) demographic variables coming afterwards.

You might notice that the names of the demographic variable values in the term variable of the simulations data all have the word “years”. We can use the str_subset() function of the stringr package to select just the demographic variables based on the word “years. In contrast, we can use the negate = TRUE argument to select all the other variables, the non-demographic variable values. We can use the unique() base function to grab just the unique values of the term variable.

str_subset(unique(pull(simulations, term)), "years", negate = TRUE)

[1] "RTC_LAWTRUE"         "Unemployment_rate"   "Poverty_rate"       
[4] "Population_log"      "police_per_100k_lag"

str_subset(unique(pull(simulations, term)), "years")

 [1] "White_Male_15_to_19_years"      "White_Male_20_to_39_years"     
 [3] "Black_Male_15_to_19_years"      "Black_Male_20_to_39_years"     
 [5] "Other_Male_15_to_19_years"      "Other_Male_20_to_39_years"     
 [7] "White_Female_10_to_19_years"    "White_Female_20_to_29_years"   
 [9] "White_Female_30_to_39_years"    "White_Female_40_to_49_years"   
[11] "White_Female_50_to_64_years"    "White_Female_65_years_and_over"
[13] "White_Male_10_to_19_years"      "White_Male_20_to_29_years"     
[15] "White_Male_30_to_39_years"      "White_Male_40_to_49_years"     
[17] "White_Male_50_to_64_years"      "White_Male_65_years_and_over"  
[19] "Black_Female_10_to_19_years"    "Black_Female_20_to_29_years"   
[21] "Black_Female_30_to_39_years"    "Black_Female_40_to_49_years"   
[23] "Black_Female_50_to_64_years"    "Black_Female_65_years_and_over"
[25] "Black_Male_10_to_19_years"      "Black_Male_20_to_29_years"     
[27] "Black_Male_30_to_39_years"      "Black_Male_40_to_49_years"     
[29] "Black_Male_50_to_64_years"      "Black_Male_65_years_and_over"  
[31] "Other_Female_10_to_19_years"    "Other_Female_20_to_29_years"   
[33] "Other_Female_30_to_39_years"    "Other_Female_40_to_49_years"   
[35] "Other_Female_50_to_64_years"    "Other_Female_65_years_and_over"
[37] "Other_Male_10_to_19_years"      "Other_Male_20_to_29_years"     
[39] "Other_Male_30_to_39_years"      "Other_Male_40_to_49_years"     
[41] "Other_Male_50_to_64_years"      "Other_Male_65_years_and_over"

Now we can create the order of the values of the term variable using the factor argument and the levels argument.

simulations <- simulations %>%
  mutate(term = factor(term,
    levels = c(
      str_subset(unique(pull(simulations, term)), "years", negate = TRUE),
      sort(str_subset(unique(pull(simulations, term)), "years"))
    )
  ))

levels(pull(simulations, term))

 [1] "RTC_LAWTRUE"                    "Unemployment_rate"             
 [3] "Poverty_rate"                   "Population_log"                
 [5] "police_per_100k_lag"            "Black_Female_10_to_19_years"   
 [7] "Black_Female_20_to_29_years"    "Black_Female_30_to_39_years"   
 [9] "Black_Female_40_to_49_years"    "Black_Female_50_to_64_years"   
[11] "Black_Female_65_years_and_over" "Black_Male_10_to_19_years"     
[13] "Black_Male_15_to_19_years"      "Black_Male_20_to_29_years"     
[15] "Black_Male_20_to_39_years"      "Black_Male_30_to_39_years"     
[17] "Black_Male_40_to_49_years"      "Black_Male_50_to_64_years"     
[19] "Black_Male_65_years_and_over"   "Other_Female_10_to_19_years"   
[21] "Other_Female_20_to_29_years"    "Other_Female_30_to_39_years"   
[23] "Other_Female_40_to_49_years"    "Other_Female_50_to_64_years"   
[25] "Other_Female_65_years_and_over" "Other_Male_10_to_19_years"     
[27] "Other_Male_15_to_19_years"      "Other_Male_20_to_29_years"     
[29] "Other_Male_20_to_39_years"      "Other_Male_30_to_39_years"     
[31] "Other_Male_40_to_49_years"      "Other_Male_50_to_64_years"     
[33] "Other_Male_65_years_and_over"   "White_Female_10_to_19_years"   
[35] "White_Female_20_to_29_years"    "White_Female_30_to_39_years"   
[37] "White_Female_40_to_49_years"    "White_Female_50_to_64_years"   
[39] "White_Female_65_years_and_over" "White_Male_10_to_19_years"     
[41] "White_Male_15_to_19_years"      "White_Male_20_to_29_years"     
[43] "White_Male_20_to_39_years"      "White_Male_30_to_39_years"     
[45] "White_Male_40_to_49_years"      "White_Male_50_to_64_years"     
[47] "White_Male_65_years_and_over"

Looks good!

Now we just need to run the same code again to create the plot, but now the order of the x axis values will be different.

simulation_plot <- simulations %>%
  ggplot(aes(x = term, y = estimate)) +
  geom_boxplot() +
  facet_grid(. ~ Analysis, scale = "free_x", space = "free", drop = TRUE) +
  labs(
    title = "Coefficient estimates",
    subtitle = "Estimates across leave-one-out analyses",
    x = "Term",
    y = "Coefficient",
    caption = "Results from simulations"
  ) +
  theme_linedraw() +
  theme(
    axis.title.x = element_blank(),
    axis.text.x = element_text(angle = 70, hjust = 1),
    strip.text.x = element_text(size = 14, face = "bold")
  )

simulation_plot

We can see that the range of coefficient estimates when only one observation is removed is much larger in Analysis 2 for nearly all variables, but particularly for many of the additional demographic variables.

Let’s make a plot showing the summary of the overall coefficient instability.

To do this we will calculate the standard deviation of coefficient estimates for each variable across all of the simulations. Thus we will group by the Analysis and the term variables now that our data is in long format. We will use the sd() function of the stats package to calculate the standard deviation.

coeff_sd <- simulations %>%
  group_by(Analysis, term) %>%
  summarize("SD" = sd(estimate))

First, we will display an interactive table of these standard deviations. Try searching for “RTC”, and you can compare the standard deviations of the coefficients for the RTC_LAWTRUE variable across the two analyses. To take a better look at our data we will use the datatable() function of the DT package which will create an interactive searchable table.

DT::datatable(coeff_sd)

Now we will make a plot of this data, including SDs from all coefficients in each model.

simulation_plot <- coeff_sd %>%
  ggplot(aes(x = Analysis, y = SD)) +
  geom_jitter(width = 0.1, alpha = 0.5, size = 2) +
  labs(
    title = "Coefficient variability",
    subtitle = "SDs of coefficient estimates from leave-one-out analysis",
    x = "Term",
    y = "Coefficient Estimate \n Standard Deviations",
    caption = "Results from simulations"
  ) +
  theme_minimal() +
  theme(
    axis.title.x = element_blank(),
    axis.text.x = element_text(size = 8, color = "black"),
    axis.text.y = element_text(color = "black")
  )
simulation_plot

Here we can clearly see that overall the coefficient estimates are much less stable in Analysis 2. This is an indication that we may have multicollinearity in our data.

VIF

Another way of evaluating the presence and severity of multicollinearity is to calculate the variance inflation factor (VIF) .

According to Wikipedia:

It provides an index that measures how much the variance (the square of the estimate’s standard deviation) of an estimated regression coefficient is increased because of collinearity.

The variance inflation factor (VIF) is the quotient of the variance in a model with multiple terms by the variance of a model with one term alone.

VIF values can be calculated for each explanatory variable in a model by performing the following calculation:

Run another ordinary least squares (OLS) linear regression with one of the explanatory variables of your model of interest ($X_i$) as the dependent variable and keep the remaining explanatory variables as explanatory variables.

So generically speaking say this is our model:

\[Y = β_0 + β_1X_1 + β_2X_2 + β_3X_3 + e \]

We have three explanatory variables ($X_1$, $X_2$, and $X_3$).

If we want to calculate the VIF value for $X_1$ we would need to perform another OLS model, where $X_1$ is now the dependent variable explained by the other explanatory variables.

\[X_1 = β_0 + β_2X_2 + β_3X_3 + e\]

The $R^2$ coefficient of determination (also called R squared value) from this regression is then used to calculate the VIF as follows:

\[\frac{1}{1-R^{2}}\]

The $R^2$ value is in this case the proportion of variance in $X_1$ explained by the other variables ($X_2$ and $X_3$).

VIF values are typically calculated for each explanatory variable when evaluating multicollinearity of a model.

The calculation for a single variable is: \[VIF_i = \frac{1}{1-R_i^{2}}\] Where $i$ is the index of each explanatory variable.

Recall that according to Wikipedia:

The variance inflation factor (VIF) is the quotient of the variance in a model with multiple terms by the variance of a model with one term alone.

The $R^2$ value ranges from 0 to 1, and if the variation of one variable is highly explained by the other variables, the $R^2$ will approach 1. Thus the denominator in the VIF calculation $1-R_i^{2}$ (which is sometimes referred to as tolerance) will be smaller and the VIF value will be larger.

Thus, higher VIF vales indicate more severe multicollinearity. Typically a threshold of a tolerance of less than 0.10 and/or a VIF of 10 or above is used as a rule of thumb to determine if the presence of multicollinearity might be problematic.

Please see this article for a thorough explanation of how to interpret VIF values and how to decide what to do if your model has high multicollinearity.

So how do we calculate VIF values in R?

We could do this manually creating many linear regressions, but that would obviously be time consuming. Luckily, the car package has a function called vif() that will calculate VIF values. However, there is one wrinkle: the vif() function is not compatible with the output of the plm function. There is however a workaround that allows us to fit a similar model using the standard lm() function on data where we have removed the within-individual means. While this won’t give us exactly the same results in terms of the standard errors of our estimates, it will give us some idea of the VIF values for the covariates in our model. We are following the steps outlined here, a really nice summary of panel data modeling in R.

Once we have calculated our VIF values, we will create nicer looking output of the data using the as_tibble() function of the tibble package to create a tibble and add the variable names as another column.

Recall that we previously created the DONOHUE_OUTPUT object like so:

DONOHUE_OUTPUT <- plm(Viol_crime_rate_1k_log ~
RTC_LAW +
  White_Male_15_to_19_years +
  White_Male_20_to_39_years +
  Black_Male_15_to_19_years +
  Black_Male_20_to_39_years +
  Other_Male_15_to_19_years +
  Other_Male_20_to_39_years +
  Unemployment_rate +
  Poverty_rate +
  Population_log +
  police_per_100k_lag,
effect = "twoways",
model = "within",
data = d_panel_DONOHUE
)

What we have modeled is how a state’s violent crime rate has changed with modifications of RTC law status and over time, relative to itself, and how this compares to similar changes in violent crime of another state relative to itself.

The coefficients from this model then are, in an oversimplified explanation, centered across all states and across all time points. This is called “demeaned” data.

We will now use this model output to create a data frame of demeaned data (where the effect of time is accounted for as are the within-individuals effects, in this case the different states). We will make a model matrix of this data by using the model.matrix() function of the stats package and then we will create a data frame from this using the base as.data.frame() function.

lm_DONOHUE_data <- as.data.frame(model.matrix(DONOHUE_OUTPUT))

glimpse(lm_DONOHUE_data)

Rows: 1,395
Columns: 11
$ RTC_LAWTRUE               <dbl> -0.1491039, -0.1491039, -0.1491039, -0.14910…
$ White_Male_15_to_19_years <dbl> -0.17167489, -0.06619055, -0.06629357, -0.08…
$ White_Male_20_to_39_years <dbl> 3.2089257, 2.8592259, 2.7093166, 2.4692997, …
$ Black_Male_15_to_19_years <dbl> -0.098689711, -0.076244940, -0.056288623, -0…
$ Black_Male_20_to_39_years <dbl> 0.211104389, 0.178202827, 0.160277326, 0.141…
$ Other_Male_15_to_19_years <dbl> 0.090970393, 0.082703650, 0.030284388, -0.04…
$ Other_Male_20_to_39_years <dbl> 0.017436309, 0.005636164, 0.023425966, -0.00…
$ Unemployment_rate         <dbl> 0.872831541, 0.203942652, -1.202724014, -1.2…
$ Poverty_rate              <dbl> -0.2531900, -1.7598566, -1.1331900, 0.615698…
$ Population_log            <dbl> -0.223047491, -0.200306695, -0.139763637, -0…
$ police_per_100k_lag       <dbl> 13.0776479, 16.2876170, 2.6545268, 140.18171…

Notice that this does not contain any outcome data. We will add this by taking the outcome of the Within() function of the plm package to get the violent crime data after accounting for the state specific effects. According to the documentation for this package:

Within returns a vector containing the values in deviation from the individual means (if effect = “individual”, from time means if effect = “time”), the so called demeaned data.

Also recall that the d_panel_DONOHUE data is just the Donohue data in panel format.

lm_DONOHUE_data %<>%
  mutate(Viol_crime_rate_1k_log = plm::Within(pull(
    d_panel_DONOHUE, Viol_crime_rate_1k_log
  )), effect = "twoways")

Now we will fit the demeaned data to the model:

lm_DONOHUE <- lm(Viol_crime_rate_1k_log ~
RTC_LAWTRUE +
  White_Male_15_to_19_years +
  White_Male_20_to_39_years +
  Black_Male_15_to_19_years +
  Black_Male_20_to_39_years +
  Other_Male_15_to_19_years +
  Other_Male_20_to_39_years +
  Unemployment_rate +
  Poverty_rate +
  Population_log +
  police_per_100k_lag,
data = lm_DONOHUE_data
)

Now we are ready to use the vif() function of the car package to calculate the VIF values:

vif_DONOHUE <- vif(lm_DONOHUE)

vif_DONOHUE

              RTC_LAWTRUE White_Male_15_to_19_years White_Male_20_to_39_years 
                 1.097853                  1.172339                  1.738459 
Black_Male_15_to_19_years Black_Male_20_to_39_years Other_Male_15_to_19_years 
                 1.344193                  1.653712                  1.586648 
Other_Male_20_to_39_years         Unemployment_rate              Poverty_rate 
                 1.529688                  1.244667                  1.270321 
           Population_log       police_per_100k_lag 
                 1.153933                  1.204491

Now we will use the as_tibble() function of the tibble package to nicely put this together.

vif_DONOHUE <- vif_DONOHUE %>%
  as_tibble() %>%
  cbind(., names(vif_DONOHUE)) %>%
  as_tibble()

colnames(vif_DONOHUE) <- c("VIF", "Variable")

vif_DONOHUE

# A tibble: 11 × 2
     VIF Variable                 
   <dbl> <chr>                    
 1  1.10 RTC_LAWTRUE              
 2  1.17 White_Male_15_to_19_years
 3  1.74 White_Male_20_to_39_years
 4  1.34 Black_Male_15_to_19_years
 5  1.65 Black_Male_20_to_39_years
 6  1.59 Other_Male_15_to_19_years
 7  1.53 Other_Male_20_to_39_years
 8  1.24 Unemployment_rate        
 9  1.27 Poverty_rate             
10  1.15 Population_log           
11  1.20 police_per_100k_lag

Now we will do the same for the Lott and Mustard data.

We will need to use the rename() function (you may recall this is part of the dplyr package) to replace RTC_LAWTRUE with RTC_LAW, as the variable name in the model output is appended by TRUE because it was a logical variable. Because the model formula for the Lott analysis is so complex, it is easier to change this variable name in our new data frame, rather than rewrite the formula for this data.

Recall that we already saved the formula for this data:

LOTT_fmla

Viol_crime_rate_1k_log ~ RTC_LAW + White_Female_10_to_19_years + 
    White_Female_20_to_29_years + White_Female_30_to_39_years + 
    White_Female_40_to_49_years + White_Female_50_to_64_years + 
    White_Female_65_years_and_over + White_Male_10_to_19_years + 
    White_Male_20_to_29_years + White_Male_30_to_39_years + White_Male_40_to_49_years + 
    White_Male_50_to_64_years + White_Male_65_years_and_over + 
    Black_Female_10_to_19_years + Black_Female_20_to_29_years + 
    Black_Female_30_to_39_years + Black_Female_40_to_49_years + 
    Black_Female_50_to_64_years + Black_Female_65_years_and_over + 
    Black_Male_10_to_19_years + Black_Male_20_to_29_years + Black_Male_30_to_39_years + 
    Black_Male_40_to_49_years + Black_Male_50_to_64_years + Black_Male_65_years_and_over + 
    Other_Female_10_to_19_years + Other_Female_20_to_29_years + 
    Other_Female_30_to_39_years + Other_Female_40_to_49_years + 
    Other_Female_50_to_64_years + Other_Female_65_years_and_over + 
    Other_Male_10_to_19_years + Other_Male_20_to_29_years + Other_Male_30_to_39_years + 
    Other_Male_40_to_49_years + Other_Male_50_to_64_years + Other_Male_65_years_and_over + 
    Unemployment_rate + Poverty_rate + Population_log + police_per_100k_lag

lm_LOTT_data <- as.data.frame(model.matrix(LOTT_OUTPUT))
lm_LOTT_data %<>%
  mutate(Viol_crime_rate_1k_log = plm::Within(pull(
    d_panel_LOTT, Viol_crime_rate_1k_log
  ), effect = "twoways")) %>%
  rename(RTC_LAW = RTC_LAWTRUE)

lm_LOTT <- lm(LOTT_fmla,
  data = lm_LOTT_data
)

vif_LOTT <- vif(lm_LOTT)

vif_LOTT

                       RTC_LAW    White_Female_10_to_19_years 
                      1.621662                     127.920555 
   White_Female_20_to_29_years    White_Female_30_to_39_years 
                     42.269637                      49.635014 
   White_Female_40_to_49_years    White_Female_50_to_64_years 
                     37.550101                      36.451868 
White_Female_65_years_and_over      White_Male_10_to_19_years 
                     12.866751                     126.824984 
     White_Male_20_to_29_years      White_Male_30_to_39_years 
                     39.248785                      73.008959 
     White_Male_40_to_49_years      White_Male_50_to_64_years 
                     31.613855                      52.774694 
  White_Male_65_years_and_over    Black_Female_10_to_19_years 
                     13.285326                     335.136906 
   Black_Female_20_to_29_years    Black_Female_30_to_39_years 
                    106.644486                      79.058455 
   Black_Female_40_to_49_years    Black_Female_50_to_64_years 
                     98.434064                      66.888057 
Black_Female_65_years_and_over      Black_Male_10_to_19_years 
                     49.715869                     320.740453 
     Black_Male_20_to_29_years      Black_Male_30_to_39_years 
                     89.297151                      89.267356 
     Black_Male_40_to_49_years      Black_Male_50_to_64_years 
                     92.498486                      64.538516 
  Black_Male_65_years_and_over    Other_Female_10_to_19_years 
                     37.960126                     142.283700 
   Other_Female_20_to_29_years    Other_Female_30_to_39_years 
                     64.966861                      54.511835 
   Other_Female_40_to_49_years    Other_Female_50_to_64_years 
                    224.567085                     131.463113 
Other_Female_65_years_and_over      Other_Male_10_to_19_years 
                     82.394398                     151.930450 
     Other_Male_20_to_29_years      Other_Male_30_to_39_years 
                     54.620045                      62.267344 
     Other_Male_40_to_49_years      Other_Male_50_to_64_years 
                    244.698473                     174.184553 
  Other_Male_65_years_and_over              Unemployment_rate 
                     53.532299                       1.497864 
                  Poverty_rate                 Population_log 
                      1.412397                       3.426475 
           police_per_100k_lag 
                      1.732745

vif_LOTT <- vif_LOTT %>%
  as_tibble() %>%
  cbind(., names(vif_LOTT)) %>%
  as_tibble()

colnames(vif_LOTT) <- c("VIF", "Variable")

Now to have consistent variable names in the VIF data sets we will rename RTC_LAW back to RTC_LAWTRUE using the str_replace() function of the stringr package. This function replaces a pattern.

vif_LOTT %>% mutate(Variable = str_replace(
  string = Variable,
  pattern = "RTC_LAW",
  replacement = "RTC_LAWTRUE"
))

# A tibble: 41 × 2
      VIF Variable                      
    <dbl> <chr>                         
 1   1.62 RTC_LAWTRUE                   
 2 128.   White_Female_10_to_19_years   
 3  42.3  White_Female_20_to_29_years   
 4  49.6  White_Female_30_to_39_years   
 5  37.6  White_Female_40_to_49_years   
 6  36.5  White_Female_50_to_64_years   
 7  12.9  White_Female_65_years_and_over
 8 127.   White_Male_10_to_19_years     
 9  39.2  White_Male_20_to_29_years     
10  73.0  White_Male_30_to_39_years     
# … with 31 more rows

vif_LOTT

# A tibble: 41 × 2
      VIF Variable                      
    <dbl> <chr>                         
 1   1.62 RTC_LAW                       
 2 128.   White_Female_10_to_19_years   
 3  42.3  White_Female_20_to_29_years   
 4  49.6  White_Female_30_to_39_years   
 5  37.6  White_Female_40_to_49_years   
 6  36.5  White_Female_50_to_64_years   
 7  12.9  White_Female_65_years_and_over
 8 127.   White_Male_10_to_19_years     
 9  39.2  White_Male_20_to_29_years     
10  73.0  White_Male_30_to_39_years     
# … with 31 more rows

DT::datatable(vif_LOTT)

We can see that some of the VIF values are very high!

Now we will make a plot of the VIF values for both analyses. We will add text to a specific location on the plot using the geom_text() function. Typically a threshold of 10 is used to identify if the VIF are problematically high.

vif_DONOHUE %<>%
  mutate(Analysis = "Analysis 1")
vif_LOTT %<>%
  mutate(Analysis = "Analysis 2")

vif_df <- bind_rows(
  vif_DONOHUE,
  vif_LOTT
)
datatable(vif_df)

You can also search the above table of results for “RTC” to see how the VIF values differ for the RTC variable between the two analyses. They are close to one another, although the value is slightly higher for Analysis 2.

Next, we will make a couple of plots to illustrate how the VIF values compare between the two models.

vif_plot <- vif_df %>%
  ggplot(aes(x = Analysis, y = VIF)) +
  geom_jitter(width = 0.1, alpha = 0.5, size = 2) +
  geom_hline(yintercept = 10, color = "red") +
  geom_text(aes(.7, 18, label = "typical cutoff of 10")) +
  labs(title = "Variance inflation factors", y = "VIF") +
  theme_minimal() +
  theme(
    axis.title.x = element_blank(),
    axis.text.x = element_text(color = "black"),
    axis.text.y = element_text(color = "black")
  )

vif_plot

We can see that analysis 2 has variables with much higher multicollinearity.

Let’s make this plot a little easier to see using the coord_trans() function of the ggplot2 package with the y ="log10" argument. This does not change the values, but adjusts the way the y axis is displayed with diminishing distance between grid lines.

vif_plot <- vif_df %>%
  ggplot(aes(x = Analysis, y = VIF)) +
  geom_jitter(width = 0.1, alpha = 0.5, size = 2) +
  geom_hline(yintercept = 10, color = "red") +
  geom_text(aes(.75, 13, label = "typical cutoff of 10")) +
  coord_trans(y = "log10") +
  labs(title = "Variance inflation factors") +
  theme_minimal() +
  theme(
    axis.title.x = element_blank(),
    axis.text.x = element_text(color = "black"),
    axis.text.y = element_text(color = "black")
  )

vif_plot

In many cases it would be advisable to remove one or more of these variables and reassess the VIF values. There are also other options, such as ridge regression. However, both of these options need to be done with care as they can also introduce bias into the model.

In any case the presence of multicollinearity should encourage further investigation about the design of the model, as the results may not be reliable due to the increased level of instability of the coefficient estimates.

See this article for a detailed discussion about what to consider when your model has variables with high VIF values.

Data Visualization

Now lets make a plot that summarizes all of our findings.

We will use the cowplot package to put our plots together.

We will use the ggdraw() function of this package. This allows you to add labels and other plot aspects on top of existing plots. Thus if we want to add a title element to our overall plot that we will add to our combined plot we can use ggdraw() to start and then the draw_label() function to add text.

[1] "gg"     "ggplot"

As you can see we know have plot object that just has the text "Multicollinearity and its effects".

Now we will create a subtitle in the same way.

forward <- ggdraw() +
  draw_label(
    "Analysis 1: 6 demographic variables\nAnalysis 2: 36 demographic variables",
    fontface = "bold",
    size = 10,
    x = 0,
    hjust = -0.02
  ) +
  theme(
    plot.margin = margin(0, 0, 0, 0)
  )

forward

Now we will recreate our correlation plots with some slight alterations. We want to remove our labels, because they will be too small to see when we combine our plots. To do this we will use the theme_void() function of the ggplot2 package.

Note that because we are layering ggplot2 objects we can’t use the %>% pipe to start with the existing correlation plots.

corr_mat_DONOHUE <- corr_mat_DONOHUE +
  theme_void() +
  theme(plot.title = element_text(size = 8, color = "black")) +
  labs(title = "Analysis 1")

corr_mat_LOTT <- corr_mat_LOTT +
  theme_void() +
  theme(plot.title = element_text(size = 8, color = "black")) +
  labs(title = "Analysis 2")

OK we want to arrange our correlation plots to be in the top row of our larger plot. Now we will use the plot_grid() function to arrange the plots.

row_A <- plot_grid(corr_mat_DONOHUE,
  corr_mat_LOTT,
  nrow = 1
)
row_A

Nice! We have combined plots!

Now let’s add a title for these plots.

title_A <- ggdraw() +
  draw_label(
    "Correlation between variables can induce multicollinearity",
    fontface = "bold",
    size = 14,
    x = 0,
    hjust = -0.01
  ) +
  theme(
    plot.margin = margin(0, 0, 0, 0)
  )

plot_A <- plot_grid(title_A,
  row_A,
  ncol = 1,
  rel_heights = c(0.1, 1)
)

For our second row in our larger plot we want to have the formula for calculating VIF values on the left and the plot that we created previously showing VIF values on the right.

First we will create a plot object that just has the formula.

To do so we are going to create a plot with a large label in the middle containing the formula. Then we will use the theme_void() function again to remove the axis labels and background.

To create our plot we will first plot values from 1-10 for both the x and y axis, allowing us to center the formula at the x and y values of 5.

To type the formula we will use LaTeX mathematical notation.

The start and end of inline mathematical formulas are specified using dollar signs ($).
Subscripts are written by using an underscore (_) and brackets ({}) indicate the start and end of the subscript.

Fractions are indicated using \frac{numerator}{denominator}.

Superscripts are created using the carrot symbol (^) and brackets ({}) indicate the start and end of the superscript.

Greek letters are created by using, for example, \beta.

In the case of the fraction and Greek letters an additional \ is needed in the Tex() function.

We will use the TeX() function of the latex2exp package to convert our LaTeX string to a plotmath expression (a mathematical notation in R to be used in plots).

empty_df <- cbind(c(1:10), c(1:10)) %>%
  as.data.frame()

colnames(empty_df) <- c("X", "Y")

plot_B1 <- ggplot(empty_df, aes(x = X, y = Y)) +
  annotate("text",
    x = 5,
    y = 8,
    label = TeX("$X_{1} = \\beta_{0} + \\beta_{2}X_{2} + \\beta_{3}X_{3}...+\\beta_{k}X_{k}+e$"),
    size = 7
  ) +
  ylim(0, 10) +
  xlim(0, 10) +
  annotate("text",
    x = 5.9,
    y = 5.5,
    label = TeX("$R_1^{2}$"),
    size = 7
  ) +
  ylim(0, 10) +
  xlim(0, 10) +
  geom_segment(aes(x = 5, y = 6, xend = 5, yend = 4.5),
    arrow = arrow(angle = 45, ends = "last", type = "open"),
    size = 1.8,
    color = "black",
    lineend = "butt",
    linejoin = "mitre"
  ) +
  annotate("text",
    x = 5,
    y = 2,
    label = TeX("$VIF_{1} = \\frac{1}{1-R_{1}^{2}}$"),
    size = 7
  )




plot_B1

plot_B1 <- plot_B1 +
  theme_void()

plot_B1

Now we will combine this with the VIF plot.

plot_B2 <- vif_plot +
  theme(axis.text.x = element_text(size = 8))

row_B <- plot_grid(plot_B1,
  plot_B2,
  nrow = 1
)

title_B <- ggdraw() +
  draw_label(
    "Variance inflation factors can be used to identify multicollinearity when present",
    fontface = "bold",
    size = 14,
    x = 0,
    hjust = -.01,
  ) +
  theme(
    plot.margin = margin(0, 0, 0, 0)
  )

plot_B <- plot_grid(title_B,
  row_B,
  ncol = 1,
  rel_heights = c(0.1, 1)
)

plot_B

Now for the third row we want to include the comparing_analyses_plot and the simulation_plot.

plot_C1 <- comparing_analyses_plot +
  theme(
    axis.text.x = element_text(size = 8),
    axis.title.x = element_blank()
  ) +
  labs(
    title = "Results in different estimates",
    subtitle = "Different demographic groupings can change direction of estimate"
  )

plot_C2 <- simulation_plot +
  labs(title = "Reduces precision in estimates")

row_C <- plot_grid(plot_C1,
  plot_C2,
  nrow = 1
)

title_C <- ggdraw() +
  draw_label(
    "Multicollinearity can have an effect on statistical inference",
    fontface = "bold",
    size = 14,
    x = 0,
    hjust = -0.01
  ) +
  theme(
    plot.margin = margin(0, 0, 0, 0)
  )

plot_C <- plot_grid(title_C,
  row_C,
  ncol = 1,
  rel_heights = c(0.1, 1)
)

plot_C

Now that we have all of our rows we can combine everything together. We will also make the background of the plot white, which would be transparent otherwise.

plots <- plot_grid(plot_A,
  plot_B,
  plot_C,
  ncol = 1,
  rel_heights = c(1, 1, 1)) 


mainplot <- plot_grid(title_plots,
  forward,
  plots,
  ncol = 1,
  rel_heights = c(
    0.05,
    0.05,
    1
  )
) +
  theme(plot.background = element_rect(fill = "white"))

mainplot

Summary

This case study has introduced the concept of multicollinearity by exploring data related to violent crimes and right-to-carry gun laws. We also introduced the topic of panel data as a special type of longitudinal data that includes data of 2 or more individuals or groups over 2 or more time points. We learned that we can use the plm package to perform panel linear regression analysis. We learned that the fixed effect model in panel analysis actually makes the least assumptions, and is therefore often the most appropriate test.

By evaluating two analyses that were identical except for the inclusion of extra demographic variables (Analysis 1 included 6, while Analysis 2 included 36), we discovered that redundant and collinear variables can change the directionality and magnitude of our findings.

We learned that by looking at the correlation between pairs of explanatory variables we can get a sense about whether multicollinearity may exist in our data.

We learned that we can evaluate the stability of our coefficient estimates across sub-samples or calculate variance inflation factor (VIF) values to get a sense of the presence and severity of multicollinearity.

We learned that often a rule of thumb of >10 is used as a threshold for raising concern about the severity of multicollinearity. However, we also learned that (as often is the case with thresholds) more care may be required.

Overall we learned that multicollinearity can bias our regression findings and it is good practice to check for multicollinearity when performing regression analysis. It is something to keep in mind when we encounter coefficient estimates that are unexpected.

Importantly this case study showcases how methodological details, like how we decide to parse our demographic variables, can have great consequences on the results of our analyses.

Suggested Homework

Ask students to remove one or more of the demographic variables with high VIF values from the Mustard and Lott-like panel data and perform the panel linear regression analysis again, as well as calculate the VIF values.

Ask the students to discuss how this possibly changed the results.

Additional Information

For more information on linear regression see this book and this case study.

For more information on the different types of panel regression models see this book, here, and here.

For more information on implementing panel regression in R using the plm package, see here and here.

For more information on multicollinearity and VIF, see this article.DOI 10.1007/s11135-006-9018-6

The articles used to motivate this case study are:
Mustard and Lott
Donohue, et al.
See here for a list of studies on this topic

For tutorials about using fixed effect models in panel analysis with the plm package, see here and here

Packages used in this case study:

Package	Use in this case study
here	to easily load and save data
dplyr	to arrange/filter/select/compare specific subsets of the data
magrittr	to use the compound assignment pipe operator `%<>%`
purrr	to import the data in all the different excel and csv files efficiently
tibble	to create data objects that we can manipulate with `dplyr`/`stringr`/`tidyr`/`purrr`
ggplot2	to create plots
ggrepel	to allow labels in figures not to overlap
plm	to work with panel data fitting fixed effects and linear regression models
broom	to create nicely formatted model output
GGally	to extend ggplot2 functionality to easily create more complex plots
ggcorrplot	to easily visualize a correlation matrix
rsample	to split our sample for the simulation analysis
DT	to create interactive and searchable tables
car	to calculate VIF values on linear model output
stringr	to manipulate the character strings within the data
cowplot	to allow plots to be combined
latex2exp	to convert latex math formulas to R’s plotmath expressions

Session Info

sessionInfo()

R version 4.2.2 (2022-10-31)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Big Sur ... 10.16

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] latex2exp_0.9.6           cowplot_1.1.1            
 [3] stringr_1.5.0             car_3.1-1                
 [5] carData_3.0-5             DT_0.23                  
 [7] rsample_1.0.0             ggcorrplot_0.1.4         
 [9] GGally_2.1.2              broom_1.0.2              
[11] plm_2.6-2                 ggrepel_0.9.2            
[13] naniar_0.6.1              ggplot2_3.4.0            
[15] tibble_3.1.8              purrr_1.0.1              
[17] dplyr_1.0.10              koRpus.lang.en_0.1-4     
[19] koRpus_0.13-8             sylly_0.1-6              
[21] OCSdata_1.1.0             read.so_0.1.1            
[23] wordcountaddin_0.3.0.9000 magrittr_2.0.3           
[25] knitr_1.41                here_1.0.1               

loaded via a namespace (and not attached):
 [1] nlme_3.1-160       fs_1.6.1           usethis_2.1.6      RColorBrewer_1.1-3
 [5] httr_1.4.5         rprojroot_2.0.3    gh_1.4.0           tools_4.2.2       
 [9] backports_1.4.1    bslib_0.4.2        utf8_1.2.2         R6_2.5.1          
[13] mgcv_1.8-41        DBI_1.1.3          colorspace_2.0-3   withr_2.5.0       
[17] tidyselect_1.2.0   curl_5.0.0         compiler_4.2.2     textshaping_0.3.6 
[21] cli_3.6.0          sandwich_3.0-2     labeling_0.4.2     sass_0.4.4        
[25] scales_1.2.1       lmtest_0.9-40      systemfonts_1.0.4  digest_0.6.31     
[29] rmarkdown_2.19     pkgconfig_2.0.3    htmltools_0.5.4    parallelly_1.32.1 
[33] fastmap_1.1.0      collapse_1.8.9     highr_0.10         htmlwidgets_1.5.4 
[37] rlang_1.0.6        rstudioapi_0.14    farver_2.1.1       jquerylib_0.1.4   
[41] generics_0.1.3     zoo_1.8-11         jsonlite_1.8.4     crosstalk_1.2.0   
[45] Formula_1.2-4      Matrix_1.5-1       Rcpp_1.0.9         munsell_0.5.0     
[49] fansi_1.0.3        abind_1.4-5        lifecycle_1.0.3    furrr_0.3.0       
[53] visdat_0.5.3       stringi_1.7.8      yaml_2.3.7         MASS_7.3-58.1     
[57] plyr_1.8.8         grid_4.2.2         parallel_4.2.2     listenv_0.8.0     
[61] crayon_1.5.2       bdsmatrix_1.3-6    lattice_0.20-45    splines_4.2.2     
[65] pillar_1.8.1       reshape2_1.4.4     codetools_0.2-18   glue_1.6.2        
[69] evaluate_0.19      data.table_1.14.6  remotes_2.4.2      vctrs_0.5.2       
[73] Rdpack_2.4         miscTools_0.6-26   gtable_0.3.1       tidyr_1.2.1       
[77] reshape_0.8.9      future_1.27.0      assertthat_0.2.1   cachem_1.0.6      
[81] xfun_0.36          sylly.en_0.1-3     rbibutils_2.2.11   ragg_1.2.2        
[85] maxLik_1.5-2       globals_0.15.1     ellipsis_0.3.2

Estimate of RMarkdown Compilation Time:

About 46 - 56 seconds

This compilation time was measured on a PC machine operating on Windows 10. This range should only be used as an estimate as compilation time will vary with different machines and operating systems.

Acknowledgments

We would like to acknowledge Daniel Webster for assisting in framing the major direction of the case study. We would also like to thank Elizabeth Stuart and Aboozar Hadavand and Alexander McCourt for reviewing the case study.

We would like to acknowledge Michael Breshock for his contributions to this case study and developing the OCSdata package.

We would also like to acknowledge the Bloomberg American Health Initiative for funding this work.

LS0tCnRpdGxlOiAiT3BlbiBDYXNlIFN0dWRpZXM6IEluZmx1ZW5jZSBvZiBNdWx0aWNvbGxpbmVhcml0eSBvbiBNZWFzdXJlZCBJbXBhY3Qgb2YgUmlnaHQtdG8tQ2FycnkgR3VuIExhd3MiCmNzczogc3R5bGUuY3NzCm91dHB1dDoKICBodG1sX2RvY3VtZW50OgogICAgaW5jbHVkZXM6CiAgICAgIGluX2hlYWRlcjogR0FfU2NyaXB0LlJodG1sCiAgICBzZWxmX2NvbnRhaW5lZDogeWVzCiAgICBjb2RlX2Rvd25sb2FkOiB5ZXMKICAgIGhpZ2hsaWdodDogdGFuZ28KICAgIG51bWJlcl9zZWN0aW9uczogbm8KICAgIHRoZW1lOiBjb3NtbwogICAgdG9jOiB5ZXMKICAgIHRvY19mbG9hdDogeWVzCiAgcGRmX2RvY3VtZW50OgogICAgdG9jOiB5ZXMKICB3b3JkX2RvY3VtZW50OgogICAgdG9jOiB5ZXMKLS0tCgo8c3R5bGU+CiNUT0MgewogIGJhY2tncm91bmQ6IHVybCgiaHR0cHM6Ly9vcGVuY2FzZXN0dWRpZXMuZ2l0aHViLmlvL2ltZy9pY29uLWJhaGkucG5nIik7CiAgYmFja2dyb3VuZC1zaXplOiBjb250YWluOwogIHBhZGRpbmctdG9wOiAyNDBweCAhaW1wb3J0YW50OwogIGJhY2tncm91bmQtcmVwZWF0OiBuby1yZXBlYXQ7Cn0KPC9zdHlsZT4KCgotLS0KCgpgYGB7ciBzZXR1cCwgaW5jbHVkZT1GQUxTRX0Ka25pdHI6Om9wdHNfY2h1bmskc2V0KAogIGluY2x1ZGUgPSBUUlVFLCBjb21tZW50ID0gTkEsIGVjaG8gPSBUUlVFLAogIG1lc3NhZ2UgPSBGQUxTRSwgd2FybmluZyA9IEZBTFNFLCBjYWNoZSA9IEZBTFNFLCBmaWcud2lkdGggPSAxMCwgZmlnLmhlaWdodCA9IDcsCiAgZmlnLmFsaWduID0gImNlbnRlciIsIG91dC53aWR0aCA9ICI5MCUiCikKbGlicmFyeShoZXJlKQpsaWJyYXJ5KGtuaXRyKQpsaWJyYXJ5KG1hZ3JpdHRyKQpyZW1vdGVzOjppbnN0YWxsX2dpdGh1Yigib3BlbmNhc2VzdHVkaWVzL09DU2RhdGEiKQpyZW1vdGVzOjppbnN0YWxsX2dpdGh1YigiYmVubWFyd2ljay93b3JkY291bnRhZGRpbiIsIHR5cGUgPSAic291cmNlIiwgZGVwZW5kZW5jaWVzID0gVFJVRSkKcmVtb3Rlczo6aW5zdGFsbF9naXRodWIoImFsaXN0YWlyZTQ3L3JlYWQuc28iKQpsaWJyYXJ5KHdvcmRjb3VudGFkZGluKQpsaWJyYXJ5KHJlYWQuc28pCmxpYnJhcnkoT0NTZGF0YSkKCnJtYXJrZG93bjo6OnBlcmZfdGltZXJfcmVzZXRfYWxsKCkKcm1hcmtkb3duOjo6cGVyZl90aW1lcl9zdGFydCgicmVuZGVyIikKYGBgCgo8ZGl2IGlkPSJnb29nbGVfdHJhbnNsYXRlX2VsZW1lbnQiPjwvZGl2PgoKPHNjcmlwdCB0eXBlPSJ0ZXh0L2phdmFzY3JpcHQiIHNyYz0nLy90cmFuc2xhdGUuZ29vZ2xlLmNvbS90cmFuc2xhdGVfYS9lbGVtZW50LmpzP2NiPWdvb2dsZVRyYW5zbGF0ZUVsZW1lbnRJbml0Jz48L3NjcmlwdD4KCjxzY3JpcHQgdHlwZT0idGV4dC9qYXZhc2NyaXB0Ij4KZnVuY3Rpb24gZ29vZ2xlVHJhbnNsYXRlRWxlbWVudEluaXQoKSB7CiAgbmV3IGdvb2dsZS50cmFuc2xhdGUuVHJhbnNsYXRlRWxlbWVudCh7cGFnZUxhbmd1YWdlOiAnZW4nfSwgJ2dvb2dsZV90cmFuc2xhdGVfZWxlbWVudCcpOwp9Cjwvc2NyaXB0PgoKCiMjIyMgey5vdXRsaW5lfQoKYGBge3IsIGVjaG8gPSBGQUxTRSwgb3V0LndpZHRoID0gIjgwMCBweCJ9CmtuaXRyOjppbmNsdWRlX2dyYXBoaWNzKGhlcmU6OmhlcmUoImltZyIsICJtYWlucGxvdC5wbmciKSkKYGBgCgojIyMjIAoKIyMjIyB7LmRpc2NsYWltZXJfYmxvY2t9CgoqKkRpc2NsYWltZXIqKjogVGhlIHB1cnBvc2Ugb2YgdGhlIFtPcGVuIENhc2UgU3R1ZGllc10oaHR0cHM6Ly9vcGVuY2FzZXN0dWRpZXMuZ2l0aHViLmlvKXt0YXJnZXQ9Il9ibGFuayJ9IHByb2plY3QgaXMgKip0byBkZW1vbnN0cmF0ZSB0aGUgdXNlIG9mIHZhcmlvdXMgZGF0YSBzY2llbmNlIG1ldGhvZHMsIHRvb2xzLCBhbmQgc29mdHdhcmUgaW4gdGhlIGNvbnRleHQgb2YgbWVzc3ksIHJlYWwtd29ybGQgZGF0YSoqLiBBIGdpdmVuIGNhc2Ugc3R1ZHkgZG9lcyBub3QgY292ZXIgYWxsIGFzcGVjdHMgb2YgdGhlIHJlc2VhcmNoIHByb2Nlc3MsIGlzIG5vdCBjbGFpbWluZyB0byBiZSB0aGUgbW9zdCBhcHByb3ByaWF0ZSB3YXkgdG8gYW5hbHl6ZSBhIGdpdmVuIGRhdGEgc2V0LCBhbmQgc2hvdWxkIG5vdCBiZSB1c2VkIGluIHRoZSBjb250ZXh0IG9mIG1ha2luZyBwb2xpY3kgZGVjaXNpb25zIHdpdGhvdXQgZXh0ZXJuYWwgY29uc3VsdGF0aW9uIGZyb20gc2NpZW50aWZpYyBleHBlcnRzLiAKCiMjIyMKCiMjIyMgey5saWNlbnNlX2Jsb2NrfQoKVGhpcyB3b3JrIGlzIGxpY2Vuc2VkIHVuZGVyIHRoZSBDcmVhdGl2ZSBDb21tb25zIEF0dHJpYnV0aW9uLU5vbkNvbW1lcmNpYWwgMy4wIFsoQ0MgQlktTkMgMy4wKV0oaHR0cHM6Ly9jcmVhdGl2ZWNvbW1vbnMub3JnL2xpY2Vuc2VzL2J5LW5jLzMuMC91cy8pe3RhcmdldD0iX2JsYW5rIn0gIFVuaXRlZCBTdGF0ZXMgTGljZW5zZS4KCiMjIyMKCiMjIyMgey5yZWZlcmVuY2VfYmxvY2t9CgpUbyBjaXRlIHRoaXMgY2FzZSBzdHVkeSBwbGVhc2UgdXNlOgoKV3JpZ2h0LCBDYXJyaWUgYW5kIE9udGl2ZXJvcywgTWljaGFlbCBhbmQgTWVuZywgUWllciBhbmQgSmFnZXIsIExlYWggYW5kIFRhdWIsIE1hcmdhcmV0IGFuZCBIaWNrcywgU3RlcGhhbmllLiAoMjAyMCkuIFtodHRwczovL2dpdGh1Yi5jb20vL29wZW5jYXNlc3R1ZGllcy9vY3MtYnAtUlRDLWFuYWx5c2lzXShodHRwczovL2dpdGh1Yi5jb20vL29wZW5jYXNlc3R1ZGllcy9vY3MtYnAtUlRDLWFuYWx5c2lzKS4gSW5mbHVlbmNlIG9mIE11bHRpY29sbGluZWFyaXR5IG9uIE1lYXN1cmVkIEltcGFjdCBvZiBSaWdodC10by1DYXJyeSBHdW4gTGF3cyAoVmVyc2lvbiB2MS4wLjApLgoKIyMjIwoKVG8gYWNjZXNzIHRoZSBHaXRIdWIgcmVwb3NpdG9yeSBmb3IgdGhpcyBjYXNlIHN0dWR5IHNlZSBoZXJlOiBodHRwczovL2dpdGh1Yi5jb20vL29wZW5jYXNlc3R1ZGllcy9vY3MtYnAtUlRDLWFuYWx5c2lzLiAgCgpZb3UgbWF5IGFsc28gYWNjZXNzIGFuZCBkb3dubG9hZCB0aGUgZGF0YSB1c2luZyBvdXIgYE9DU2RhdGFgIHBhY2thZ2UuIFRvIGxlYXJuIG1vcmUgYWJvdXQgdGhpcyBwYWNrYWdlIGluY2x1ZGluZyBleGFtcGxlcywgc2VlIHRoaXMgW2xpbmtdKGh0dHBzOi8vZ2l0aHViLmNvbS9vcGVuY2FzZXN0dWRpZXMvT0NTZGF0YSkuIEhlcmUgaXMgaG93IHlvdSB3b3VsZCBpbnN0YWxsIHRoaXMgcGFja2FnZToKCmBgYHtyLCBldmFsPUZBTFNFfQppbnN0YWxsLnBhY2thZ2VzKCJPQ1NkYXRhIikKYGBgCgpUaGlzIGNhc2Ugc3R1ZHkgaXMgcGFydCBvZiBhIHNlcmllcyBvZiBwdWJsaWMgaGVhbHRoIGNhc2Ugc3R1ZGllcyBmb3IgdGhlIFtCbG9vbWJlcmcgQW1lcmljYW4gSGVhbHRoIEluaXRpYXRpdmVdKGh0dHBzOi8vYW1lcmljYW5oZWFsdGguamh1LmVkdS9vcGVuLWNhc2Utc3R1ZGllcykuCgoqKioKClRoZSB0b3RhbCByZWFkaW5nIHRpbWUgZm9yIHRoaXMgY2FzZSBzdHVkeSBpcyBjYWxjdWxhdGVkIHZpYSBba29ScHVzXShodHRwczovL2dpdGh1Yi5jb20vdW5Eb2NVTWVhbnRJdC9rb1JwdXMpIGFuZCBzaG93biBiZWxvdzogCgpgYGB7ciwgZWNobz1GQUxTRX0KcmVhZHRhYmxlID0gdGV4dF9zdGF0cygiaW5kZXguUm1kIikgIyBwcm9kdWNpbmcgcmVhZGluZyB0aW1lIG1hcmtkb3duIHRhYmxlCnJlYWR0aW1lID0gcmVhZC5zbzo6cmVhZC5tZChyZWFkdGFibGUpICU+JSBkcGx5cjo6c2VsZWN0KE1ldGhvZCwga29ScHVzKSAlPiUgIyByZWFkaW5nIHRhYmxlIGludG8gZGF0YWZyYW1lLCBzZWxlY3RpbmcgcmVsZXZhbnQgZmFjdG9ycwogIGRwbHlyOjpmaWx0ZXIoTWV0aG9kID09ICJSZWFkaW5nIHRpbWUiKSAlPiUgIyBkcm9wcGluZyB1bm5lY2Vzc2FyeSByb3dzCiAgZHBseXI6Om11dGF0ZShrb1JwdXMgPSBwYXN0ZShyb3VuZChhcy5udW1lcmljKHN0cmluZ3I6OnN0cl9zcGxpdChrb1JwdXMsICIgIilbWzFdXVsxXSkpLCAibWludXRlcyIpKSAlPiUgIyByb3VuZGluZyByZWFkaW5nIHRpbWUgZXN0aW1hdGUKICBkcGx5cjo6bXV0YXRlKE1ldGhvZCA9ICJrb1JwdXMiKSAlPiUgZHBseXI6OnJlbG9jYXRlKGtvUnB1cywgLmJlZm9yZSA9IE1ldGhvZCkgJT4lIGRwbHlyOjpyZW5hbWUoYFJlYWRpbmcgVGltZWAgPSBrb1JwdXMpICMgcmVvcmdhbml6aW5nIHRhYmxlCmtuaXRyOjprYWJsZShyZWFkdGltZSwgZm9ybWF0PSJtYXJrZG93biIpCmBgYAoKKioqCgoqKlJlYWRhYmlsaXR5IFNjb3JlOiAqKgoKQSByZWFkYWJpbGl0eSBpbmRleCBlc3RpbWF0ZXMgdGhlIHJlYWRpbmcgZGlmZmljdWx0eSBsZXZlbCBvZiBhIHBhcnRpY3VsYXIgdGV4dC4gRmxlc2NoLUtpbmNhaWQsIEZPUkNBU1QsIGFuZCBTTU9HIGFyZSB0aHJlZSBjb21tb24gcmVhZGFiaWxpdHkgaW5kaWNlcyB0aGF0IHdlcmUgY2FsY3VsYXRlZCBmb3IgdGhpcyBjYXNlIHN0dWR5IHZpYSBba29ScHVzXShodHRwczovL2dpdGh1Yi5jb20vdW5Eb2NVTWVhbnRJdC9rb1JwdXMpLiBUaGVzZSBpbmRpY2VzIHByb3ZpZGUgYW4gZXN0aW1hdGlvbiBvZiB0aGUgbWluaW11bSByZWFkaW5nIGxldmVsIHJlcXVpcmVkIHRvIGNvbXByZWhlbmQgdGhpcyBjYXNlIHN0dWR5IGJ5IGdyYWRlIGFuZCBhZ2UuIAoKYGBge3IsIGVjaG89RkFMU0V9CnJ0ID0gd29yZGNvdW50YWRkaW46OnJlYWRhYmlsaXR5KCJpbmRleC5SbWQiLCBxdWlldD1UUlVFKSAjIHByb2R1Y2luZyByZWFkYWJpbGl0eSBtYXJrZG93biB0YWJsZQpkZiA9IHJlYWQuc286OnJlYWQubWQocnQpICU+JSBkcGx5cjo6c2VsZWN0KGluZGV4LCBncmFkZSwgYWdlKSAlPiUgICMgcmVhZGluZyB0YWJsZSBpbnRvIGRhdGFmcmFtZSwgc2VsZWN0aW5nIHJlbGV2YW50IGZhY3RvcnMKICB0aWR5cjo6ZHJvcF9uYSgpICU+JSBkcGx5cjo6bXV0YXRlKGdyYWRlID0gcm91bmQoYXMubnVtZXJpYyhncmFkZSkpLCAjIGRyb3BwaW5nIHJvd3Mgd2l0aCBtaXNzaW5nIHZhbHVlcywgcm91bmRpbmcgYWdlIGFuZCBncmFkZSBjb2x1bW5zCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBhZ2UgPSByb3VuZChhcy5udW1lcmljKGFnZSkpCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICApCmtuaXRyOjprYWJsZShkZiwgZm9ybWF0PSJtYXJrZG93biIpCmBgYAoKKioqCgpQbGVhc2UgaGVscCB1cyBieSBmaWxsaW5nIG91dCBvdXIgc3VydmV5LgoKCjxkaXYgc3R5bGU9ImRpc3BsYXk6IGZsZXg7IGp1c3RpZnktY29udGVudDogY2VudGVyOyI+PGlmcmFtZSBzcmM9Imh0dHBzOi8vZG9jcy5nb29nbGUuY29tL2Zvcm1zL2QvZS8xRkFJcFFMU2ZwTjRGTjNLRUxxQk5FZ2YyQXRwaTdXeTdOcXkyYmVTa0ZRSU5MN1k1c0FNVjVfdy92aWV3Zm9ybT9lbWJlZGRlZD10cnVlIiB3aWR0aD0iMTIwMCIgaGVpZ2h0PSI3MDAiIGZyYW1lYm9yZGVyPSIwIiBtYXJnaW5oZWlnaHQ9IjAiIG1hcmdpbndpZHRoPSIwIj5Mb2FkaW5n4oCmPC9pZnJhbWU+PC9kaXY+CgoKCiMgKipNb3RpdmF0aW9uKioKKioqIAoKVGhpcyBjYXNlIHN0dWR5IHdpbGwgaW50cm9kdWNlIHRoZSB0b3BpYyBvZiBbbXVsdGljb2xsaW5lYXJpdHldKGh0dHBzOi8vZW4ud2lraXBlZGlhLm9yZy93aWtpL011bHRpY29sbGluZWFyaXR5KXt0YXJnZXQ9Il9ibGFuayJ9LCB3aGljaCBvY2N1cnMgaW4gcmVncmVzc2lvbiB3aGVuIG9uZSBvciBtb3JlIGluZGVwZW5kZW50IHZhcmlhYmxlcyBjYW4gYmUgcHJlZGljdGVkIGJ5IG90aGVyIGluZGVwZW5kZW50IHZhcmlhYmxlcy4gCgpXZSB3aWxsIGRvIHNvIGJ5IHNob3djYXNpbmcgYSByZWFsIHdvcmxkIGV4YW1wbGUgd2hlcmUgbXVsdGljb2xsaW5lYXJpdHkgaW4gcGFydCByZXN1bHRlZCBpbiBoaXN0b3JpY2FsbHkgY29udHJvdmVyc2lhbCBhbmQgY29uZmxpY3RpbmcgZmluZGluZ3MgYWJvdXQgdGhlIGluZmx1ZW5jZSBvZiB0aGUgYWRvcHRpb24gb2YgcmlnaHQtdG8tY2FycnkgKFJUQykgY29uY2VhbGVkIGhhbmRndW4gbGF3cyBvbiB2aW9sZW50IGNyaW1lIHJhdGVzIGluIHRoZSBVbml0ZWQgU3RhdGVzLiAKCldlIHdpbGwgZm9jdXMgb24gdHdvIGFydGljbGVzOgoKMS4gVGhlIGZpcnN0IGFuYWx5c2lzIGJ5IFtNdXN0YXJkIGFuZCBMb3R0XShodHRwczovL2NoaWNhZ291bmJvdW5kLnVjaGljYWdvLmVkdS9jZ2kvdmlld2NvbnRlbnQuY2dpP2FydGljbGU9MTE1MCZjb250ZXh0PWxhd19hbmRfZWNvbm9taWNzKXt0YXJnZXQ9Il9ibGFuayJ9IHB1Ymxpc2hlZCBpbiAxOTk2IHN1Z2dlc3RzIHRoYXQgUlRDIGxhd3MgcmVkdWNlIHZpb2xlbnQgY3JpbWUuIExvdHQgYXV0aG9yZWQgYSBib29rIGV4dGVuZGluZyB0aGVzZSBmaW5kaW5ncyBpbiAxOTk4IGNhbGxlZCBbKioqTW9yZSBHdW5zLCBMZXNzIENyaW1lKioqXShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9Nb3JlX0d1bnMsX0xlc3NfQ3JpbWUpe3RhcmdldD0iX2JsYW5rIn0uCgpgYGB7ciwgZWNobz1GQUxTRSwgb3V0LmhlaWdodCA9ICcxMDAlJywgb3V0LndpZHRoID0gJzEwMCUnLCBmaWcuYWxpZ249J2NlbnRlcid9CmtuaXRyOjppbmNsdWRlX2dyYXBoaWNzKGhlcmUoImltZyIsICJMb3R0LnBuZyIpKQpgYGAKCiMjIyMjIFtbc291cmNlXV0oaHR0cHM6Ly9jaGljYWdvdW5ib3VuZC51Y2hpY2Fnby5lZHUvY2dpL3ZpZXdjb250ZW50LmNnaT9hcnRpY2xlPTExNTAmY29udGV4dD1sYXdfYW5kX2Vjb25vbWljcyl7dGFyZ2V0PSJfYmxhbmsifQoKMi4gVGhlIHNlY29uZCBhbmFseXNpcyBpcyBhIHJlY2VudCBhcnRpY2xlIGJ5IFtEb25vaHVlLCBldCBhbC5dKGh0dHBzOi8vd3d3Lm5iZXIub3JnL3BhcGVycy93MjM1MTAucGRmKXt0YXJnZXQ9Il9ibGFuayJ9IHB1Ymxpc2hlZCBpbiAyMDE3IHRoYXQgc3VnZ2VzdHMgdGhhdCBSVEMgbGF3cyBpbmNyZWFzZSB2aW9sZW50IGNyaW1lLiBEb25vaHVlIGhhcyBhbHNvIHB1Ymxpc2hlZCBwcmV2aW91cyBhcnRpY2xlcyB3aXRoIHRpdGxlcyBzdWNoIGFzIFsqKipTaG9vdGluZyBkb3duIHRoZSAiTW9yZSBHdW5zLCBMZXNzIENyaW1lIiBIeXBvdGhlc2lzKioqXShodHRwczovL3d3dy5qc3Rvci5vcmcvc3RhYmxlLzEyMjk2MDM/c2VxPTEpe3RhcmdldD0iX2JsYW5rIn0uIAoKYGBge3IsIGVjaG89RkFMU0UsIG91dC5oZWlnaHQgPSAnMTAwJScsIG91dC53aWR0aCA9ICcxMDAlJywgZmlnLmFsaWduPSdjZW50ZXInfQprbml0cjo6aW5jbHVkZV9ncmFwaGljcyhoZXJlKCJpbWciLCAiRG9ub2h1ZS5wbmciKSkKYGBgCgojIyMjIyBbW3NvdXJjZV1dKGh0dHBzOi8vd3d3Lm5iZXIub3JnL3BhcGVycy93MjM1MTAucGRmKXt0YXJnZXQ9Il9ibGFuayJ9CgpUaGlzIGhhcyBiZWVuIGEgY29udHJvdmVyc2lhbCB0b3BpYyBhcyBtYW55IG90aGVyIGFuYWx5c2VzIGFsc28gcHJvZHVjZWQgY29uZmxpY3RpbmcgcmVzdWx0cy4gU2VlIFtoZXJlXShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9Nb3JlX0d1bnMsX0xlc3NfQ3JpbWUpe3RhcmdldD0iX2JsYW5rIn0gZm9yIGEgbGlzdCBvZiBzdHVkaWVzLgoKVGhlIFtEb25vaHVlLCBldCBhbC5dKGh0dHBzOi8vd3d3Lm5iZXIub3JnL3BhcGVycy93MjM1MTAucGRmKXt0YXJnZXQ9Il9ibGFuayJ9IGFydGljbGUgZGlzY3Vzc2VzIGhvdyB0aGVyZSBhcmUgbWFueSBvdGhlciBpbXBvcnRhbnQgbWV0aG9kb2xvZ2ljYWwgYXNwZWN0cyBiZXNpZGVzIFttdWx0aWNvbGxpbmVhcml0eV0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvTXVsdGljb2xsaW5lYXJpdHkpe3RhcmdldD0iX2JsYW5rIn0gKHdoaWNoIG9jY3VycyB3aGVuIHByZWRpY3RvciBvciBpbnB1dCB2YXJpYWJsZXMgYXJlIGhpZ2hseSByZWxhdGVkIGluIGEgcmVncmVzc2lvbiBhbmFseXNpcykgdGhhdCBjb3VsZCBhY2NvdW50IGZvciB0aGUgaGlzdG9yaWNhbGx5IGNvbmZsaWN0aW5nIHJlc3VsdHMgaW4gdGhlc2UgcHJldmlvdXMgbWFudXNjcmlwdHMuCgpJbiBmYWN0LCBuZWFybHkgZXZlcnkgYXNwZWN0IG9mIHRoZSBkYXRhIGFuYWx5c2lzIHByb2Nlc3Mgd2FzIGRpZmZlcmVudCBiZXR3ZWVuIHRoZSBbRG9ub2h1ZSwgZXQgYWwuXShodHRwczovL3d3dy5uYmVyLm9yZy9wYXBlcnMvdzIzNTEwLnBkZil7dGFyZ2V0PSJfYmxhbmsifSBhbmQgW011c3RhcmQgYW5kIExvdHRdKGh0dHBzOi8vY2hpY2Fnb3VuYm91bmQudWNoaWNhZ28uZWR1L2NnaS92aWV3Y29udGVudC5jZ2k/YXJ0aWNsZT0xMTUwJmNvbnRleHQ9bGF3X2FuZF9lY29ub21pY3Mpe3RhcmdldD0iX2JsYW5rIn0gYW5hbHlzZXMuCgpgYGB7ciwgZWNobz1GQUxTRSwgb3V0LmhlaWdodCA9ICc3NSUnLCBvdXQud2lkdGggPSAnNzUlJywgZmlnLmFsaWduPSdjZW50ZXInfQprbml0cjo6aW5jbHVkZV9ncmFwaGljcyhoZXJlKCJpbWciLCAiRWR1Y2F0aW9uYWxfR3JhcGhpYzEuanBnIikpCmBgYAoKCkhvd2V2ZXIsIHdlIHdpbGwgZm9jdXMgcGFydGljdWxhcmx5IG9uIG11bHRpY29sbGluZWFyaXR5IGFuZCBob3cgaXQgY2FuIGluZmx1ZW5jZSB0aGUgcmVzdWx0cyB3ZSBnZXQgZnJvbSBsaW5lYXIgcmVncmVzc2lvbi4gClNwZWNpZmljYWxseSwgdGhpcyBhbmFseXNpcyB3aWxsIGRlbW9uc3RyYXRlIGhvdyBtZXRob2RvbG9naWNhbCBkZXRhaWxzIGNhbiBiZSBjcml0aWNhbGx5IGluZmx1ZW50aWFsIGZvciBvdXIgb3ZlcmFsbCBjb25jbHVzaW9ucyBhbmQgY2FuIHJlc3VsdCBpbiBpbXBvcnRhbnQgcG9saWN5IHJlbGF0ZWQgY29uc2VxdWVuY2VzLiBUaGUgW0Rvbm9odWUsIGV0IGFsLiBhcnRpY2xlXSgoaHR0cHM6Ly93d3cubmJlci5vcmcvcGFwZXJzL3cyMzUxMC5wZGYpe3RhcmdldD0iX2JsYW5rIn0pIHdpbGwgcHJvdmlkZSB0aGUgbW90aXZhdGlvbiBhbmQgaWxsdXN0cmF0aW9uLiAKCiMjIyMgey5yZWZlcmVuY2VfYmxvY2t9CgpKb2huIEouIERvbm9odWUgZXQgYWwuLCBSaWdodOKAkHRv4oCQQ2FycnkgTGF3cyBhbmQgVmlvbGVudCBDcmltZTogQSBDb21wcmVoZW5zaXZlIEFzc2Vzc21lbnQgVXNpbmcgUGFuZWwgRGF0YSBhbmQgYSBTdGF0ZeKAkExldmVsIFN5bnRoZXRpYyBDb250cm9sIEFuYWx5c2lzLiAqSm91cm5hbCBvZiBFbXBpcmljYWwgTGVnYWwgU3R1ZGllcyosIDE2LDIgKDIwMTkpLgoKRGF2aWQgQi4gTXVzdGFyZCAmIEpvaG4gTG90dC4gQ3JpbWUsIERldGVycmVuY2UsIGFuZCBSaWdodC10by1DYXJyeSBDb25jZWFsZWQgSGFuZGd1bnMuICpDb2FzZS1TYW5kb3IgSW5zdGl0dXRlIGZvciBMYXcgJiBFY29ub21pY3MqIFdvcmtpbmcgUGFwZXIgTm8uIDQxLCAoMTk5NikuCgojIyMjCgoKQmVmb3JlIHdlIGxlYXZlIHRoaXMgc2VjdGlvbiwgd2UgcHJvdmlkZSBhIGhpZ2gtbGV2ZWwgb3ZlcnZpZXcgb2Ygd2hhdCB2YXJpYWJsZXMgd2VyZSAob3Igd2VyZSBub3QpIGluY2x1ZGVkIGluIHRoZSBbRG9ub2h1ZSwgQW5lamEgYW5kIFdlYmVyXShodHRwczovL3d3dy5uYmVyLm9yZy9wYXBlcnMvdzIzNTEwLnBkZil7dGFyZ2V0PSJfYmxhbmsifSAoREFXKSBwYXBlciBhbmQgdGhlIFtNdXN0YXJkIGFuZCBMb3R0XShodHRwczovL2NoaWNhZ291bmJvdW5kLnVjaGljYWdvLmVkdS9jZ2kvdmlld2NvbnRlbnQuY2dpP2FydGljbGU9MTE1MCZjb250ZXh0PWxhd19hbmRfZWNvbm9taWNzKXt0YXJnZXQ9Il9ibGFuayJ9IChNTCkgcGFwZXI6CgoKYGBge3IsIGVjaG89RkFMU0UsIG91dC5oZWlnaHQgPSAnMTAwJScsIG91dC53aWR0aCA9ICcxMDAlJywgZmlnLmFsaWduPSdjZW50ZXInfQprbml0cjo6aW5jbHVkZV9ncmFwaGljcyhoZXJlKCJpbWciLCAiRG9ub2h1ZV9UYWJsZTJfZWRpdGVkLnBuZyIpKQpgYGAKCiMjIyMjIFtbc291cmNlXV0oaHR0cHM6Ly93d3cubmJlci5vcmcvcGFwZXJzL3cyMzUxMC5wZGYpe3RhcmdldD0iX2JsYW5rIn0KCgojIyMjIyMgKk1MIGlzIGFiYnJldmlhdGVkIGFzIExNIGluIHRoZSBzb3VyY2UgYXJ0aWNsZQoKKipOb3RlKio6IFdlIGFyZSBub3QgYXR0ZW1wdGluZyB0byByZS1jcmVhdGUgdGhlIGFuYWx5c2VzIGZyb20gdGhlIG9yaWdpbmFsIGF1dGhvcnMuIEluc3RlYWQsIHdlIGFpbSB0byB1c2UgYSBzdWJzZXQgb2YgdGhlIGxpc3RlZCBleHBsYW5hdG9yeSB2YXJpYWJsZXMgaW4gdGhpcyBjYXNlIHN0dWR5IHRvIGRlbW9uc3RyYXRlIG11bHRpY29sbGluZWFyaXR5LiBUaGVzZSB2YXJpYWJsZXMgd2lsbCBiZSBjb25zaXN0ZW50IGZvciBib3RoIGFuYWx5c2VzIHRoYXQgd2Ugd2lsbCBwZXJmb3JtLCB3aXRoIHRoZSBleGNlcHRpb24gdGhhdCBvbmUgYW5hbHlzaXMgd2lsbCBoYXZlIDYgZGVtb2dyYXBoaWMgdmFyaWFibGVzIGFzIGluIHRoZSBhbmFseXNpcyBpbiB0aGUgW0Rvbm9odWUsIGV0IGFsLl0oaHR0cHM6Ly93d3cubmJlci5vcmcvcGFwZXJzL3cyMzUxMC5wZGYpe3RhcmdldD0iX2JsYW5rIn0gYXJ0aWNsZSBhbmQgdGhlIG90aGVyIHdpbGwgaGF2ZSAzNiBkZW1vZ3JhcGhpYyB2YXJpYWJsZXMsIGdyb3VwaW5nIGluZGl2aWR1YWxzIGludG8gbW9yZSBzcGVjaWZpYyBjYXRlZ29yaWVzLCBhcyBpbiB0aGUgYW5hbHlzaXMgaW4gdGhlIFtNdXN0YXJkIGFuZCBMb3R0XShodHRwczovL2NoaWNhZ291bmJvdW5kLnVjaGljYWdvLmVkdS9jZ2kvdmlld2NvbnRlbnQuY2dpP2FydGljbGU9MTE1MCZjb250ZXh0PWxhd19hbmRfZWNvbm9taWNzKXt0YXJnZXQ9Il9ibGFuayJ9IGFydGljbGUuCgoKIyAqKk1haW4gUXVlc3Rpb24qKgoqKiogCgojIyMjIHsubWFpbl9xdWVzdGlvbl9ibG9ja30KPGI+PHU+IE91ciBtYWluIHF1ZXN0aW9uOiA8L3U+PC9iPgoKV2hhdCBpcyB0aGUgZWZmZWN0IG9mIG11bHRpY29sbGluZWFyaXR5IG9uIGNvZWZmaWNpZW50IGVzdGltYXRlcyBmcm9tIGxpbmVhciByZWdyZXNzaW9uIG1vZGVscyB3aGVuIGFuYWx5emluZyByaWdodCB0byBjYXJyeSBsYXdzIGFuZCB2aW9sZW5jZSByYXRlcz8KCiMjIyMKCgpTcGVjaWZpY2FsbHksIHdlIHdpbGwgY29uc2lkZXIgdGhlIHR3byB3YXlzIHRvIGRlZmluZSB0aGUgZGVtb2dyYXBoaWMgdmFyaWFibGVzIChhcyBkZXNjcmliZWQgYWJvdmUpIGFuZCBpbnZlc3RpZ2F0ZSBob3cgdGhlIGluY2x1c2lvbiBvZiBkaWZmZXJlbnQgbnVtYmVycyBvZiBhZ2UgZ3JvdXBzIGluZmx1ZW5jZXMgdGhlIHJlc3VsdHMgb2YgYW4gYW5hbHlzaXMgb2YgcmlnaHQgdG8gY2FycnkgbGF3cyBhbmQgdmlvbGVuY2UgcmF0ZXMuCgoKIyAqKkxlYXJuaW5nIE9iamVjdGl2ZXMqKiAKKioqIAoKVGhlIHNraWxscywgbWV0aG9kcywgYW5kIGNvbmNlcHRzIHRoYXQgc3R1ZGVudHMgd2lsbCBiZSBmYW1pbGlhciB3aXRoIGJ5IHRoZSBlbmQgb2YgdGhpcyBjYXNlIHN0dWR5IGFyZToKCjx1PioqRGF0YSBTY2llbmNlIExlYXJuaW5nIE9iamVjdGl2ZXM6Kio8L3U+ICAKCjEuIENyZWF0ZSBjb3JyZWxhdGlvbiBzY2F0dGVycGxvdHMgYW5kIGhlYXRtYXBzIChgR0dhbGx5YCwgYGdnY29ycnBsb3RgKSAgCjIuIENyZWF0ZSBpbnRlcmFjdGl2ZSB0YWJsZXMgKGBEVGApICAKMy4gU2FtcGxlIHN1YnNldHMgb2YgZGF0YSAoYHJzYW1wbGVgKSAgCjQuIENvbWJpbmUgbXVsdGlwbGUgcGxvdHMgKGBjb3dwbG90YCkgIAo1LiBDcmVhdGUgZGF0YSB2aXN1YWxpemF0aW9ucyB3aXRoIGVxdWF0aW9ucyBhbmQgdGV4dChgZ2dwbG90MmAgYW5kIGBsYXRleDJleHBgKSAgCgo8dT4qKlN0YXRpc3RpY2FsIExlYXJuaW5nIE9iamVjdGl2ZXM6Kio8L3U+IAoKMS4gVW5kZXJzdGFuZCB3aGF0IG11bHRpY29sbGluZWFyaXR5IGlzIGFuZCBob3cgaXQgY2FuIGluZmx1ZW5jZSBsaW5lYXIgcmVncmVzc2lvbiBjb2VmZmljaWVudHMKMi4gUmVjb2duaXplIHNpZ25zIGZvciB0aGUgcHJlc2VuY2Ugb2YgbXVsdGljb2xsaW5lYXJpdHkgYW5kIGRldGVybWluZSBpdHMgc2V2ZXJpdHkKMy4gSWxsdXN0cmF0ZSB0aGUgZGlmZmVyZW5jZSBiZXR3ZWVuIG11bHRpY29sbGluZWFyaXR5IGFuZCBjb3JyZWxhdGlvbiAKNC4gSW1wbGVtZW50IHBhbmVsIHJlZ3Jlc3Npb24gYW5hbHlzaXMgaW4gUiAoYHBsbWApCjUuIFJlbGF0ZSB2YXJpYW5jZSBpbmZsYXRpb24gZmFjdG9ycyAoVklGcykgdG8gbXVsdGljb2xsaW5lYXJpdHkgYW5kIGNhbGN1bGF0ZSBWSUZzIGluIFIgKGBjYXJgKQoKVG8gc2VlIGFub3RoZXIgY2FzZSBzdHVkeSBhYm91dCBob3cgdGhlIG9yaWdpbmFsIHJhdyBkYXRhIHdhcyBpbXBvcnRlZCBhbmQgd3JhbmdsZWQgcGxlYXNlIHNlZSBbaGVyZV0oaHR0cHM6Ly93d3cub3BlbmNhc2VzdHVkaWVzLm9yZy9vY3MtYnAtUlRDLXdyYW5nbGluZy8pe3RhcmdldD0iX2JsYW5rIn0uCgpXZSB3aWxsIGVzcGVjaWFsbHkgZm9jdXMgb24gdXNpbmcgcGFja2FnZXMgYW5kIGZ1bmN0aW9ucyBmcm9tIHRoZSBbYHRpZHl2ZXJzZWBdKGh0dHBzOi8vd3d3LnRpZHl2ZXJzZS5vcmcvKXt0YXJnZXQ9Il9ibGFuayJ9LCBzdWNoIGFzIGBkcGx5cmAgYW5kIGBnZ3Bsb3QyYC4gVGhlIHRpZHl2ZXJzZSBpcyBhIGxpYnJhcnkgb2YgcGFja2FnZXMgY3JlYXRlZCBieSBSU3R1ZGlvLiBXaGlsZSBzb21lIHN0dWRlbnRzIG1heSBiZSBmYW1pbGlhciB3aXRoIHByZXZpb3VzIFIgcHJvZ3JhbW1pbmcgcGFja2FnZXMsIHRoZXNlIHBhY2thZ2VzIG1ha2UgZGF0YSBzY2llbmNlIGluIFIgZXNwZWNpYWxseSBsZWdpYmxlIGFuZCBpbnR1aXRpdmUuCgpgYGB7ciwgb3V0LndpZHRoID0gIjIwJSIsIGVjaG8gPSBGQUxTRSwgZmlnLmFsaWduID0iY2VudGVyIn0KaW5jbHVkZV9ncmFwaGljcygiaHR0cHM6Ly90aWR5dmVyc2UudGlkeXZlcnNlLm9yZy9sb2dvLnBuZyIpCmBgYAoKIyAqKkNvbnRleHQqKgoqKioKClNvIHdoYXQgZXhhY3RseSBpcyBhICoqcmlnaHQtdG8tY2FycnkgbGF3Kio/CgpJdCBpcyBhIGxhdyB0aGF0IHNwZWNpZmllcyBfaWZfIGFuZCBfaG93XyBjaXRpemVucyBhcmUgYWxsb3dlZCB0byBoYXZlIGEgZmlyZWFybSBvbiB0aGVpciBwZXJzb24gb3IgbmVhcmJ5IChmb3IgZXhhbXBsZSwgaW4gYSBjaXRpemVuJ3MgY2FyKSBpbiBwdWJsaWMuIEluIHRoaXMgZGlzY3Vzc2lvbiwgd2Ugd2lsbCB1c2UgdGhlIFtOYXRpb25hbCBSaWZsZSBBc3NvY2lhdGlvbiAoTlJBKV0oaHR0cHM6Ly93d3cubnJhaWxhLm9yZy9ndW4tbGF3cy8pe3RhcmdldD0iX2JsYW5rIn0gdGVybWlub2xvZ3kuIFBsZWFzZSBrZWVwIGluIG1pbmQgdGhhdCB0aGVyZSBhcmUgb3RoZXIgdGVybXMgdGhhdCBwZW9wbGUgdXNlLiAKClRoZSBbU2Vjb25kIEFtZW5kbWVudF0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvU2Vjb25kX0FtZW5kbWVudF90b190aGVfVW5pdGVkX1N0YXRlc19Db25zdGl0dXRpb24pe3RhcmdldD0iX2JsYW5rIn0gdG8gdGhlIFVuaXRlZCBTdGF0ZXMgQ29uc3RpdHV0aW9uIGd1YXJhbnRlZXMgdGhlIHJpZ2h0IHRvICJrZWVwIGFuZCBiZWFyIGFybXMiLiBUaGUgYW1lbmRtZW50IHdhcyByYXRpZmllZCBpbiAxNzkxIGFzIHBhcnQgb2YgdGhlIFtCaWxsIG9mIFJpZ2h0c10oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvVW5pdGVkX1N0YXRlc19CaWxsX29mX1JpZ2h0cyl7dGFyZ2V0PSJfYmxhbmsifS4KCmBgYHtyLCBlY2hvPUZBTFNFLCBvdXQuaGVpZ2h0ID0gJzUwJScsIG91dC53aWR0aCA9ICc1MCUnLCBmaWcuYWxpZ249J2NlbnRlcid9CmtuaXRyOjppbmNsdWRlX2dyYXBoaWNzKCJodHRwczovL3VwbG9hZC53aWtpbWVkaWEub3JnL3dpa2lwZWRpYS9jb21tb25zLzcvNzkvQmlsbF9vZl9SaWdodHNfUGcxb2YxX0FDLmpwZyIpCmBgYAoKIyMjIyMgW1tzb3VyY2VdXShodHRwczovL3VwbG9hZC53aWtpbWVkaWEub3JnL3dpa2lwZWRpYS9jb21tb25zLzcvNzkvQmlsbF9vZl9SaWdodHNfUGcxb2YxX0FDLmpwZyl7dGFyZ2V0PSJfYmxhbmsifQoKSG93ZXZlciwgdGhlcmUgYXJlIG5vIGZlZGVyYWwgbGF3cyBhYm91dCBjYXJyeWluZyBmaXJlYXJtcyBpbiBwdWJsaWMuIAoKVGhlc2UgbGF3cyBhcmUgY3JlYXRlZCBhbmQgZW5mb3JjZWQgYXQgdGhlIFVTIHN0YXRlIGxldmVsLiAKU3RhdGVzIHZhcnkgZ3JlYXRseSBpbiB0aGVpciBsYXdzIGFib3V0IHRoZSByaWdodCB0byBjYXJyeSBmaXJlYXJtcy4gClNvbWUgcmVxdWlyZSBleHRlbnNpdmUgZWZmb3J0IHRvIG9idGFpbiBhIHBlcm1pdCB0byBsZWdhbGx5IGNhcnJ5IGEgZmlyZWFybSwgd2hpbGUgb3RoZXIgc3RhdGVzIHJlcXVpcmUgdmVyeSBtaW5pbWFsIGVmZm9ydCB0byBkbyBzby4gQW4gaW5jcmVhc2luZyBudW1iZXIgb2Ygc3RhdGVzIGRvIG5vdCByZXF1aXJlIHBlcm1pdHMgYXQgYWxsLgoKPGRldGFpbHM+IDxzdW1tYXJ5PiBDbGljayBoZXJlIGZvciBtb3JlIGluZm9ybWF0aW9uIG9uIGhpc3Rvcnkgb2YgcmlnaHQtdG8tY2FycnkgcG9saWNpZXMgaW4gdGhlIFVTLiA8L3N1bW1hcnk+CgpBY2NvcmRpbmcgdG8gdGhlIFtXaWtpcGVkaWEgZW50cnldKGh0dHBzOi8vZW4ud2lraXBlZGlhLm9yZy93aWtpL0hpc3Rvcnlfb2ZfY29uY2VhbGVkX2NhcnJ5X2luX3RoZV9VLlMuKXt0YXJnZXQ9Il9ibGFuayJ9IGFib3V0IHRoZSBoaXN0b3J5IG9mIHJpZ2h0LXRvLWNhcnJ5IHBvbGljaWVzIGluIHRoZSBVbml0ZWQgU3RhdGVzOgoKPiBQdWJsaWMgcGVyY2VwdGlvbiBvbiBjb25jZWFsZWQgY2FycnkgdnMgb3BlbiBjYXJyeSBoYXMgbGFyZ2VseSBmbGlwcGVkLiBJbiB0aGUgZWFybHkgZGF5cyBvZiB0aGUgVW5pdGVkIFN0YXRlcywgb3BlbiBjYXJyeWluZyBvZiBmaXJlYXJtcywgbG9uZyBndW5zIGFuZCByZXZvbHZlcnMgd2FzIGEgY29tbW9uIGFuZCB3ZWxsLWFjY2VwdGVkIHByYWN0aWNlLiBTZWVpbmcgZ3VucyBjYXJyaWVkIG9wZW5seSB3YXMgbm90IGNvbnNpZGVyZWQgdG8gYmUgYW55IGNhdXNlIGZvciBhbGFybS4gVGhlcmVmb3JlLCBhbnlvbmUgd2hvIHdvdWxkIGNhcnJ5IGEgZmlyZWFybSBidXQgYXR0ZW1wdCB0byBjb25jZWFsIGl0IHdhcyBjb25zaWRlcmVkIHRvIGhhdmUgc29tZXRoaW5nIHRvIGhpZGUsIGFuZCBwcmVzdW1lZCB0byBiZSBhIGNyaW1pbmFsLiBGb3IgdGhpcyByZWFzb24sIGNvbmNlYWxlZCBjYXJyeSB3YXMgZGVub3VuY2VkIGFzIGEgZGV0ZXN0YWJsZSBwcmFjdGljZSBpbiB0aGUgZWFybHkgZGF5cyBvZiB0aGUgVW5pdGVkIFN0YXRlcy4KCj4gQ29uY2VhbGVkIHdlYXBvbnMgYmFucyB3ZXJlIHBhc3NlZCBpbiBLZW50dWNreSBhbmQgTG91aXNpYW5hIGluIDE4MTMuIChJbiB0aG9zZSBkYXlzIG9wZW4gY2Fycnkgb2Ygd2VhcG9ucyBmb3Igc2VsZi1kZWZlbnNlIHdhcyBjb25zaWRlcmVkIGFjY2VwdGFibGU7IGNvbmNlYWxlZCBjYXJyeSB3YXMgZGVub3VuY2VkIGFzIHRoZSBwcmFjdGljZSBvZiBjcmltaW5hbHMuKSBCeSAxODU5LCBJbmRpYW5hLCBUZW5uZXNzZWUsIFZpcmdpbmlhLCBBbGFiYW1hLCBhbmQgT2hpbyBoYWQgZm9sbG93ZWQgc3VpdC4gQnkgdGhlIGVuZCBvZiB0aGUgbmluZXRlZW50aCBjZW50dXJ5LCBzaW1pbGFyIGxhd3Mgd2VyZSBwYXNzZWQgaW4gcGxhY2VzIHN1Y2ggYXMgVGV4YXMsIEZsb3JpZGEsIGFuZCBPa2xhaG9tYSwgd2hpY2ggcHJvdGVjdGVkIHNvbWUgZ3VuIHJpZ2h0cyBpbiB0aGVpciBzdGF0ZSBjb25zdGl0dXRpb25zLiBCZWZvcmUgdGhlIG1pZCAxOTAwcywgbW9zdCBVLlMuIHN0YXRlcyBoYWQgcGFzc2VkIGNvbmNlYWxlZCBjYXJyeSBsYXdzIHJhdGhlciB0aGFuIGJhbm5pbmcgd2VhcG9ucyBjb21wbGV0ZWx5LiBVbnRpbCB0aGUgbGF0ZSAxOTkwcywgbWFueSBTb3V0aGVybiBzdGF0ZXMgd2VyZSBlaXRoZXIgIk5vLUlzc3VlIiBvciAiUmVzdHJpY3RpdmUgTWF5LUlzc3VlIi4gU2luY2UgdGhlbiwgdGhlc2Ugc3RhdGVzIGhhdmUgbGFyZ2VseSBlbmFjdGVkICJTaGFsbC1Jc3N1ZSIgbGljZW5zaW5nIGxhd3MsIHdpdGggbnVtZXJvdXMgc3RhdGVzIGxlZ2FsaXppbmcgIlVucmVzdHJpY3RlZCBjb25jZWFsZWQgY2FycnkiLgoKPC9kZXRhaWxzPgoKVGhlcmUgYXJlIGZpdmUgYnJvYWQgY2F0ZWdvcmllcyBvZiByaWdodC10by1jYXJyeSBsYXdzIGFjY29yZGluZyB0byB0aGUgTlJBOgoKYGBge3IsIGVjaG89RkFMU0UsIG91dC5oZWlnaHQgPSAnMTAwJScsIG91dC53aWR0aCA9ICcxMDAlJywgZmlnLmFsaWduPSdjZW50ZXInfQprbml0cjo6aW5jbHVkZV9ncmFwaGljcyhoZXJlKCJpbWciLCAiUlRDLnBuZyIpKQpgYGAKCiMjIyMjIFtbc291cmNlXV0oaHR0cHM6Ly93d3cubnJhaWxhLm9yZy9ndW4tbGF3cy8pe3RhcmdldD0iX2JsYW5rIn0KCmBgYHtyLCBlY2hvPUZBTFNFLCBvdXQuaGVpZ2h0ID0gJzEwMCUnLCBvdXQud2lkdGggPSAnMTAwJScsIGZpZy5hbGlnbj0nY2VudGVyJ30Ka25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MoaGVyZSgiaW1nIiwgIlJUQ19tYXAucG5nIikpCmBgYAoKIyMjIyMgW1tzb3VyY2VdXShodHRwczovL3d3dy5ucmFpbGEub3JnL2d1bi1sYXdzLyl7dGFyZ2V0PSJfYmxhbmsifQoKWW91IGNhbiBzZWUgdGhhdCBubyBzdGF0ZSBpbiB0aGUgVVMgY3VycmVudGx5ICh0aGlzIG1hcCBpcyBmcm9tIDIwMjApIGhhcyBhICJSaWdodHMgSW5mcmluZ2VkL05vbi1Jc3N1ZSIgbGF3ICh0aGUgZ3JheSBjYXRlZ29yeSkgLS0gbWVhbmluZyB0aGF0IGFsbCA1MCBzdGF0ZXMgaW4gdGhlIFVTIGFsbG93IHRoZSByaWdodCB0byBjYXJyeSBmaXJlYXJtcyBhdCBsZWFzdCBpbiBzb21lIHdheS4gCkhvd2V2ZXIgdGhlIGxldmVsIG9mIHJlc3RyaWN0aW9ucyBpcyBkcmFtYXRpY2FsbHkgZGlmZmVyZW50IGZyb20gb25lIHN0YXRlIHRvIGFub3RoZXIuCkZvciBjb21wYXJpc29uIHB1cnBvc2VzLCBtYW55IGF1dGhvcnMgdXNlIHRoZSB0ZXJtcyAic2hhbGwgaXNzdWUiLCAic2hhbGwgaXNzdWUgd2l0aCBkaXNjcmV0aW9uIiwgIm5vIHBlcm1pdCByZXF1aXJlZCIsICJtYXkgaXNzdWUiLCBhbmQgIm5vbi1pc3N1ZSIgdG8gZGVzY3JpYmUgdGhlc2UgY2F0ZWdvcmllcyBpbnN0ZWFkIG9mIHRoZSBOUkEgdGVybWlub2xvZ3kuCgoKPGRldGFpbHM+IDxzdW1tYXJ5PiBDbGljayBoZXJlIGZvciBtb3JlIGluZm9ybWF0aW9uIGFib3V0IGhvdyByZXN0cmljdGlvbnMgdmFyeSBmcm9tIG9uZSBzdGF0ZSB0byBhbm90aGVyLiA8L3N1bW1hcnk+CgpUaGVyZSBpcyB2YXJpYXRpb24gZnJvbSBzdGF0ZSB0byBzdGF0ZSBldmVuIHdpdGhpbiB0aGUgc2FtZSBnZW5lcmFsIGNhdGVnb3J5OgoKRm9yIGV4YW1wbGUgaGVyZSBpcyBhbiBhYnJpZGdlZCB2ZXJzaW9uIG9mIHRoZSBbY3VycmVudCBjYXJyeSBsYXdzIGluIElkYWhvXShodHRwczovL3d3dy5ucmFpbGEub3JnL2d1bi1sYXdzL3N0YXRlLWd1bi1sYXdzL2lkYWhvLykgd2hpY2ggaXMgY29uc2lkZXJlZCBhbiAiVW5yZXN0cmljdGVkIC0gbm8gcGVybWl0IHJlcXVpcmVkIiBzdGF0ZToKCj4gU3RhdGUgbGF3IC4uLiBhbGxvd3MgYW55IHJlc2lkZW50IG9mIElkYWhvIG9yIGEgY3VycmVudCBtZW1iZXIgb2YgdGhlIGFybWVkIGZvcmNlcyBvZiB0aGUgVW5pdGVkIFN0YXRlcyB0byBjYXJyeSBhIGNvbmNlYWxlZCBoYW5kZ3VuIHdpdGhvdXQgYSBsaWNlbnNlIHRvIGNhcnJ5LCBwcm92aWRlZCB0aGUgcGVyc29uIGlzIG92ZXIgMTggeWVhcnMgb2xkIGFuZCBub3QgZGlzcXVhbGlmaWVkIGZyb20gYmVpbmcgaXNzdWVkIGEgbGljZW5zZSB0byBjYXJyeSBjb25jZWFsZWQgd2VhcG9ucyB1bmRlciBzdGF0ZSBsYXcuIEFuIGFtZW5kbWVudCB0byBzdGF0ZSBsYXcgdGhhdCB0YWtlcyBlZmZlY3Qgb24gSnVseSAxLCAyMDIwIGNoYW5nZXMgdGhlIHJlZmVyZW5jZSBpbiB0aGUgYWJvdmUgbGF3IGZyb20g4oCcYSByZXNpZGVudCBvZiBJZGFob+KAnSB0byDigJxhbnkgY2l0aXplbiBvZiB0aGUgVW5pdGVkIFN0YXRlcy7igJ0gIAoKCkFuZCBoZXJlIGFyZSBpcyBhbiBhYnJpZGdlZCB2ZXJzaW9uIG9mIHRoZSBbY3VycmVudCBjYXJyeSBsYXdzIGluIEFyaXpvbmFdKGh0dHBzOi8vd3d3Lm5yYWlsYS5vcmcvZ3VuLWxhd3Mvc3RhdGUtZ3VuLWxhd3MvYXJpem9uYS8pIHdoaWNoIGlzIGFsc28gY29uc2lkZXJlZCBhbiAiVW5yZXN0cmljdGVkIC0gbm8gcGVybWl0IHJlcXVpcmVkIiBzdGF0ZToKCj4gQW55IHBlcnNvbiAyMSB5ZWFycyBvZiBhZ2Ugb3Igb2xkZXIsIHdobyBpcyBub3QgcHJvaGliaXRlZCBwb3NzZXNzb3IsIG1heSBjYXJyeSBhIHdlYXBvbiBvcGVubHkgb3IgY29uY2VhbGVkIHdpdGhvdXQgdGhlIG5lZWQgZm9yIGEgbGljZW5zZS4uLgoKTm90aWNlIHRoYXQgY2l0aXplbnMgaW4gSWRhaG8gb25seSBuZWVkIHRvIGJlIDE4IHRvIGNhcnJ5IGEgZmlyZWFybSwgd2hlcmVhcyB0aGV5IG11c3QgYmUgMjEgaW4gQXJpem9uYS4gCgo8L2RldGFpbHM+CgoKIyAqKkxpbWl0YXRpb25zKioKKioqIAoKVGhlcmUgYXJlIHNvbWUgaW1wb3J0YW50IGNvbnNpZGVyYXRpb25zIHJlZ2FyZGluZyB0aGlzIGRhdGEgYW5hbHlzaXMgdG8ga2VlcCBpbiBtaW5kOiAKCjEuIFdlIGRvIG5vdCB1c2UgYWxsIG9mIHRoZSBkYXRhIHVzZWQgYnkgZWl0aGVyIHRoZSBbTXVzdGFyZCBhbmQgTG90dF0oaHR0cHM6Ly9jaGljYWdvdW5ib3VuZC51Y2hpY2Fnby5lZHUvY2dpL3ZpZXdjb250ZW50LmNnaT9hcnRpY2xlPTExNTAmY29udGV4dD1sYXdfYW5kX2Vjb25vbWljcyl7dGFyZ2V0PSJfYmxhbmsifSBvciBbRG9ub2h1ZSwgZXQgYWwuXShodHRwczovL3d3dy5uYmVyLm9yZy9wYXBlcnMvdzIzNTEwLnBkZil7dGFyZ2V0PSJfYmxhbmsifSBhbmFseXNlcywgbm9yIGRvIHdlIHBlcmZvcm0gdGhlIHNhbWUgYW5hbHlzaXMgYXMgaW4gZWFjaCBhcnRpY2xlLiBXZSBpbnN0ZWFkIHBlcmZvcm0gYSBtdWNoIHNpbXBsZXIgYW5hbHlzaXMgd2l0aCBmZXdlciB2YXJpYWJsZXMgZm9yIHRoZSBwdXJwb3NlcyBvZiBpbGx1c3RyYXRpb24gb2YgdGhlIGNvbmNlcHQgb2YgbXVsdGljb2xsaW5lYXJpdHkgYW5kIGl0cyBpbmZsdWVuY2Ugb24gcmVncmVzc2lvbiBjb2VmZmljaWVudHMsIG5vdCB0byByZXByb2R1Y2UgZWl0aGVyIGFuYWx5c2lzLgoKMi4gT3VyIGFuYWx5c2lzIGFjY291bnRzIGZvciBlaXRoZXIgdGhlIGFkb3B0aW9uIG9yIGxhY2sgb2YgYWRvcHRpb24gb2YgYSBwZXJtaXNzaXZlIHJpZ2h0LXRvLWNhcnJ5IGxhdyBpbiBlYWNoIHN0YXRlLCBidXQgZG9lcyBub3QgYWNjb3VudCBmb3IgZGlmZmVyZW5jZXMgaW4gdGhlIGxldmVsIG9mIHBlcm1pc3NpdmVuZXNzIG9mIHRoZSBsYXdzLgoKUmVjYWxsIHRoYXQgdGhlc2UgYXJlIHRoZSBjYXRlZ29yaWVzIG9mIHJpZ2h0IHRvIGNhcnJ5IGxhd3M6CmBgYHtyLCBlY2hvPUZBTFNFLCBvdXQuaGVpZ2h0ID0gJzEwMCUnLCBvdXQud2lkdGggPSAnMTAwJScsIGZpZy5hbGlnbj0nY2VudGVyJ30Ka25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MoaGVyZSgiaW1nIiwgIlJUQy5wbmciKSkKYGBgCgpTdGF0ZXMgd2l0aCBsYXdzIG9mIHRoZSBjYXRlZ29yeSByaWdodHMgcmVzdHJpY3RlZCAtIHZlcnkgbGltaXRlZCBpc3N1ZSAocmVkKSBhcmUgY29uc2lkZXJlZCBhcyBub3QgaGF2aW5nIGEgcGVybWlzc2l2ZSByaWdodC10by1jYXJyeSBsYXcuIFJlY2FsbCB0aGF0IG5vIHN0YXRlcyBjdXJyZW50bHkgaGF2ZSBhIHJpZ2h0cyBpbmZyaW5nZWQvbm9uLWlzc3VlIGxhdy4KClN0YXRlcyBvZiBhbGwgb3RoZXIgY2F0ZWdvcmllcyAoc2hhbGwgaXNzdWUsIGRpc2NyZXRpb25hcnkvcmVhc29uYWJsZSBpc3N1ZSwgYW5kIG5vIHBlcm1pdCByZXF1aXJlZCwgYWxsIHNoYWRlcyBvZiBibHVlKSBhcmUgY29uc2lkZXJlZCB0aGUgc2FtZSBpbiBvdXIgYW5hbHlzaXMsIGFzIGhhdmluZyBhIHBlcm1pc3NpdmUgcmlnaHQtdG8tY2FycnkgbGF3LgoKMykgQmVjYXVzZSBvdXIgYW5hbHlzaXMgaXMgYW4gb3ZlcnNpbXBsaWZpY2F0aW9uLCB0aGUgcmVzdWx0cyBwcmVzZW50ZWQgaGVyZSBzaG91bGQgbm90IGJlIHVzZWQgZm9yIGRldGVybWluaW5nIHBvbGljeSBjaGFuZ2VzOyBpbnN0ZWFkIHdlIHN1Z2dlc3QgdGhhdCB1c2VycyBpbnRlcmVzdGVkIGluIHN1Y2ggYSBkZXRlcm1pbmF0aW9uIGNvbnN1bHQgd2l0aCBhIHNwZWNpYWxpc3QuCgoKNCkgVGhlIGluY2x1c2lvbiBvZiByYWNlIGFzIGFuIGV4cGxhbmF0b3J5IHZhcmlhYmxlIGluIGFuIGVwaWRlbWlvbG9naWNhbCBzdHVkeSBjYW4gYmUgdXNlZnVsIGluIGNlcnRhaW4gY2lyY3Vtc3RhbmNlcy4gSG93ZXZlciwgdGhlcmUgYXJlIGxpbWl0YXRpb25zIGFuZCBpc3N1ZXMgYXJvdW5kIGRlZmluaW5nLCBkZXRlcm1pbmluZywgYW5kIHJlcG9ydGluZyByYWNlLCBhcyB3ZWxsIGFzIGluIGludGVycHJldGluZyBkaWZmZXJlbmNlcyBpbiBwdWJsaWMgaGVhbHRoIG91dGNvbWVzIGJ5IHJhY2UuIEZvciBtb3JlIGluZm9ybWF0aW9uIG9uIHRoaXMgdG9waWMsIHdlIGhhdmUgaW5jbHVkZWQgYSBbbGlua10oaHR0cHM6Ly9hY2FkZW1pYy5vdXAuY29tL2VwaXJldi9hcnRpY2xlLzIyLzIvMTg3LzQ1Njk0MikgdG8gYSBwYXBlciBvbiB0aGUgdXNlIG9mIHJhY2UgYXMgYSBtZWFzdXJlIGluIGVwaWRlbWlvbG9neS4gV2UgaW5jbHVkZSByYWNlIGluIHRoaXMgYW5hbHlzaXMgdG8gZGVtb25zdHJhdGUgYW5kIGNvbnNpZGVyIHRoZSBsaW1pdGF0aW9ucyBvZiB3aGF0IHRoZSBwcmV2aW91cyBwYXBlcnMgaGF2ZSBkb25lIHRvIGFuYWx5emUgdGhlIGluZmx1ZW5jZSBvZiBSVEMgbGF3cyBvbiB2aW9sZW50IGNyaW1lLCB3aXRoIGEgZm9jdXMgb24gbXVsdGljb2xsaW5lYXJpdHkuIFRodXMgaW4gb3VyIGFuYWx5c2lzIHdlIGhhdmUgYWxzbyBkZWZpbmVkIHJhY2UgYXMgd2FzIHByZXZpb3VzbHkgZG9uZSBpbiB0aGVzZSBwYXBlcnMuIEZ1cnRoZXJtb3JlLCB3ZSB3YW50IHRvIHBvaW50IG91dCB0aGF0IHJlcG9ydGluZyBhbmFseXNlcyBhYm91dCBjcmltZSB3aXRoIHJhY2UgYXMgYSB2YXJpYWJsZSBjYW4gaGF2ZSB2ZXJ5IHVuZXhwZWN0ZWQgY29uc2VxdWVuY2VzIGFuZCB0aHVzIGNhcmUgc2hvdWxkIGJlIHRha2VuLiBTZWUgW2hlcmVdKGh0dHBzOi8vam91cm5hbHMuc2FnZXB1Yi5jb20vZG9pL2Z1bGwvMTAuMTE3Ny8wOTYzNzIxNDE4NzYzOTMxKSBmb3Igc3VnZ2VzdGlvbnMuIEFueSBhc3NvY2lhdGlvbiBiZXR3ZWVuIGRlbW9ncmFwaGljIHZhcmlhYmxlcyAoaW5kaWNhdGluZyB0aGUgcHJvcG9ydGlvbiBvZiB0aGUgcG9wdWxhdGlvbiBmcm9tIHNwZWNpZmljIHJhY2UgYW5kIGFnZSBncm91cHMpIGFuZCB2aW9sZW50IGNyaW1lIGRvZXMgbm90IG5lY2Vzc2FyaWx5IGluZGljYXRlIHRoYXQgdGhlIHR3byBhcmUgbGlua2VkIGNhdXNhbGx5LCBhcyBhc2lkZSBmcm9tIHRoZSBpc3N1ZXMgcHJlc2VudGVkIGluIHRoZSBbYXJ0aWNsZV0oKGh0dHBzOi8vYWNhZGVtaWMub3VwLmNvbS9lcGlyZXYvYXJ0aWNsZS8yMi8yLzE4Ny80NTY5NDIpKSwgdGhpcyBtYXkgaW5zdGVhZCBpbmRpY2F0ZSBoaWdoZXIgcmF0ZXMgb2YgcG9saWNlIGVuZ2FnZW1lbnQgd2l0aCBjZXJ0YWluIHJhY2lhbCBncm91cHMgZHVlIHRvIFtyYWNpYWwgcHJvZmxpbmddKGh0dHBzOi8vd3d3LmFjbHUub3JnL290aGVyL3JhY2lhbC1wcm9maWxpbmctZGVmaW5pdGlvbikuCgpUaGUgQUNMVSBkZWZpbmVzIHJhY2lhbCBwcm9maWxpbmcgYXM6Cgo+IlJhY2lhbCBQcm9maWxpbmciIHJlZmVycyB0byB0aGUgZGlzY3JpbWluYXRvcnkgcHJhY3RpY2UgYnkgbGF3IGVuZm9yY2VtZW50IG9mZmljaWFscyBvZiB0YXJnZXRpbmcgaW5kaXZpZHVhbHMgZm9yIHN1c3BpY2lvbiBvZiBjcmltZSBiYXNlZCBvbiB0aGUgaW5kaXZpZHVhbCdzIHJhY2UsIGV0aG5pY2l0eSwgcmVsaWdpb24gb3IgbmF0aW9uYWwgb3JpZ2luLgoKKioqCgoKCldlIHdpbGwgYmVnaW4gYnkgbG9hZGluZyB0aGUgcGFja2FnZXMgdGhhdCB3ZSB3aWxsIG5lZWQ6CgpgYGB7cn0KbGlicmFyeShoZXJlKQpsaWJyYXJ5KGRwbHlyKQpsaWJyYXJ5KG1hZ3JpdHRyKQpsaWJyYXJ5KHB1cnJyKQpsaWJyYXJ5KHRpYmJsZSkKbGlicmFyeShnZ3Bsb3QyKQpsaWJyYXJ5KG5hbmlhcikKbGlicmFyeShnZ3JlcGVsKQpsaWJyYXJ5KHBsbSkKbGlicmFyeShicm9vbSkKbGlicmFyeShHR2FsbHkpCmxpYnJhcnkoZ2djb3JycGxvdCkKbGlicmFyeShyc2FtcGxlKQpsaWJyYXJ5KERUKQpsaWJyYXJ5KGNhcikKbGlicmFyeShzdHJpbmdyKQpsaWJyYXJ5KGNvd3Bsb3QpCmxpYnJhcnkobGF0ZXgyZXhwKQpsaWJyYXJ5KE9DU2RhdGEpCmBgYAoKPHU+KipQYWNrYWdlcyB1c2VkIGluIHRoaXMgY2FzZSBzdHVkeToqKiA8L3U+ICAKCiAgUGFja2FnZSAgIHwgVXNlIGluIHRoaXMgY2FzZSBzdHVkeSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAotLS0tLS0tLS0tIHwtLS0tLS0tLS0tLS0tCltgaGVyZWBdKGh0dHBzOi8vZ2l0aHViLmNvbS9qZW5ueWJjL2hlcmVfaGVyZSl7dGFyZ2V0PSJfYmxhbmsifSAgICAgICB8IHRvIGVhc2lseSBsb2FkIGFuZCBzYXZlIGRhdGEKW2BkcGx5cmBdKGh0dHBzOi8vZHBseXIudGlkeXZlcnNlLm9yZy8pe3RhcmdldD0iX2JsYW5rIn0gICAgICB8IHRvIGFycmFuZ2UvZmlsdGVyL3NlbGVjdC9jb21wYXJlIHNwZWNpZmljIHN1YnNldHMgb2YgdGhlIGRhdGEgIApbYG1hZ3JpdHRyYF0oaHR0cHM6Ly9jcmFuLnItcHJvamVjdC5vcmcvd2ViL3BhY2thZ2VzL21hZ3JpdHRyL3ZpZ25ldHRlcy9tYWdyaXR0ci5odG1sKXt0YXJnZXQ9Il9ibGFuayJ9IHwgdG8gdXNlIHRoZSBjb21wb3VuZCBhc3NpZ25tZW50IHBpcGUgb3BlcmF0b3IgYCU8PiVgCltgcHVycnJgXShodHRwczovL3B1cnJyLnRpZHl2ZXJzZS5vcmcvKXt0YXJnZXQ9Il9ibGFuayJ9ICAgfCB0byBpbXBvcnQgdGhlIGRhdGEgaW4gYWxsIHRoZSBkaWZmZXJlbnQgZXhjZWwgYW5kIGNzdiBmaWxlcyBlZmZpY2llbnRseSAgCltgdGliYmxlYF0oaHR0cHM6Ly90aWJibGUudGlkeXZlcnNlLm9yZy8pe3RhcmdldD0iX2JsYW5rIn0gICAgIHwgdG8gY3JlYXRlIGRhdGEgb2JqZWN0cyB0aGF0IHdlIGNhbiBtYW5pcHVsYXRlIHdpdGggYGRwbHlyYC9gc3RyaW5ncmAvYHRpZHlyYC9gcHVycnJgICAKW2BnZ3Bsb3QyYF0oaHR0cHM6Ly9nZ3Bsb3QyLnRpZHl2ZXJzZS5vcmcvKXt0YXJnZXQ9Il9ibGFuayJ9ICAgICB8IHRvIGNyZWF0ZSBwbG90cyAKW2BuYW5pYXJgXShodHRwczovL2NyYW4uci1wcm9qZWN0Lm9yZy93ZWIvcGFja2FnZXMvbmFuaWFyL3ZpZ25ldHRlcy9nZXR0aW5nLXN0YXJ0ZWQtdy1uYW5pYXIuaHRtbCl7dGFyZ2V0PSJfYmxhbmsifSAgfCB0byBxdWlja2x5IHZpc3VhbGl6ZSBtaXNzaW5nIGRhdGEgICAgIApbYGdncmVwZWxgXShodHRwczovL2NyYW4uci1wcm9qZWN0Lm9yZy93ZWIvcGFja2FnZXMvZ2dyZXBlbC92aWduZXR0ZXMvZ2dyZXBlbC5odG1sKXt0YXJnZXQ9Il9ibGFuayJ9ICAgIHwgdG8gYWxsb3cgbGFiZWxzIGluIGZpZ3VyZXMgbm90IHRvIG92ZXJsYXAgIApbYHBsbWBdKGh0dHBzOi8vY3Jhbi5yLXByb2plY3Qub3JnL3dlYi9wYWNrYWdlcy9wbG0vdmlnbmV0dGVzL3BsbVBhY2thZ2UuaHRtbCl7dGFyZ2V0PSJfYmxhbmsifSB8IHRvIHdvcmsgd2l0aCBwYW5lbCBkYXRhIGZpdHRpbmcgZml4ZWQgZWZmZWN0cyBhbmQgbGluZWFyIHJlZ3Jlc3Npb24gbW9kZWxzICAKW2Bicm9vbWBdKGh0dHBzOi8vY3Jhbi5yLXByb2plY3Qub3JnL3dlYi9wYWNrYWdlcy9icm9vbS92aWduZXR0ZXMvYnJvb20uaHRtbCl7dGFyZ2V0PSJfYmxhbmsifSB8IHRvIGNyZWF0ZSBuaWNlbHkgZm9ybWF0dGVkIG1vZGVsIG91dHB1dCAgCltgR0dhbGx5YF0oaHR0cHM6Ly9naXRodWIuY29tL2dnb2JpL2dnYWxseSl7dGFyZ2V0PSJfYmxhbmsifSB8IHRvIGV4dGVuZCBnZ3Bsb3QyIGZ1bmN0aW9uYWxpdHkgdG8gZWFzaWx5IGNyZWF0ZSBtb3JlIGNvbXBsZXggcGxvdHMgIApbYGdnY29ycnBsb3RgXShodHRwczovL3d3dy5yZG9jdW1lbnRhdGlvbi5vcmcvcGFja2FnZXMvZ2djb3JycGxvdC92ZXJzaW9ucy8wLjEuMyl7dGFyZ2V0PSJfYmxhbmsifSB8IHRvIGVhc2lseSB2aXN1YWxpemUgYSBjb3JyZWxhdGlvbiBtYXRyaXggIApbYHJzYW1wbGVgXShodHRwczovL3JzYW1wbGUudGlkeW1vZGVscy5vcmcpe3RhcmdldD0iX2JsYW5rIn0gfCB0byBzcGxpdCBvdXIgc2FtcGxlIGZvciB0aGUgc2ltdWxhdGlvbiBhbmFseXNpcyAgIApbYERUYF0oaHR0cHM6Ly9yc3R1ZGlvLmdpdGh1Yi5pby9EVC8pe3RhcmdldD0iX2JsYW5rIn0gIHwgdG8gY3JlYXRlIGludGVyYWN0aXZlIGFuZCBzZWFyY2hhYmxlIHRhYmxlcyAgCltgY2FyYF0oaHR0cHM6Ly9jcmFuLnItcHJvamVjdC5vcmcvd2ViL3BhY2thZ2VzL2Nhci92aWduZXR0ZXMvZW1iZWRkaW5nLnBkZil7dGFyZ2V0PSJfYmxhbmsifSAgfCB0byBjYWxjdWxhdGUgVklGIHZhbHVlcyBvbiBsaW5lYXIgbW9kZWwgb3V0cHV0ICAKW2BzdHJpbmdyYF0oaHR0cHM6Ly9zdHJpbmdyLnRpZHl2ZXJzZS5vcmcvYXJ0aWNsZXMvc3RyaW5nci5odG1sKXt0YXJnZXQ9Il9ibGFuayJ9ICAgIHwgdG8gbWFuaXB1bGF0ZSB0aGUgY2hhcmFjdGVyIHN0cmluZ3Mgd2l0aGluIHRoZSBkYXRhICAKW2Bjb3dwbG90YF0oaHR0cHM6Ly9jcmFuLnItcHJvamVjdC5vcmcvd2ViL3BhY2thZ2VzL2Nvd3Bsb3QvdmlnbmV0dGVzL2ludHJvZHVjdGlvbi5odG1sKXt0YXJnZXQ9Il9ibGFuayJ9IHwgdG8gYWxsb3cgcGxvdHMgdG8gYmUgY29tYmluZWQgCltgbGF0ZXgyZXhwYF0oaHR0cHM6Ly9jcmFuLnItcHJvamVjdC5vcmcvd2ViL3BhY2thZ2VzL2xhdGV4MmV4cC92aWduZXR0ZXMvdXNpbmctbGF0ZXgyZXhwLmh0bWwpe3RhcmdldD0iX2JsYW5rIn0gfCB0byBjb252ZXJ0IGxhdGV4IG1hdGggZm9ybXVsYXMgdG8gUidzIHBsb3RtYXRoIGV4cHJlc3Npb25zICAKW2BPQ1NkYXRhYF0oaHR0cHM6Ly9naXRodWIuY29tL29wZW5jYXNlc3R1ZGllcy9PQ1NkYXRhKXt0YXJnZXQ9Il9ibGFuayJ9IHwgdG8gYWNjZXNzIGFuZCBkb3dubG9hZCBPQ1MgZGF0YSBmaWxlcwoKVGhlIGZpcnN0IHRpbWUgd2UgdXNlIGEgZnVuY3Rpb24sIHdlIHdpbGwgdXNlIHRoZSBgOjpgIHRvIGluZGljYXRlIHdoaWNoIHBhY2thZ2Ugd2UgYXJlIHVzaW5nLiBVbmxlc3Mgd2UgaGF2ZSBvdmVybGFwcGluZyBmdW5jdGlvbiBuYW1lcywgdGhpcyBpcyBub3QgbmVjZXNzYXJ5LCBidXQgd2Ugd2lsbCBpbmNsdWRlIGl0IGhlcmUgdG8gYmUgaW5mb3JtYXRpdmUgYWJvdXQgd2hlcmUgdGhlIGZ1bmN0aW9ucyB3ZSB3aWxsIHVzZSBjb21lIGZyb20uCgoKIyAqKldoYXQgYXJlIHRoZSBkYXRhPyoqCioqKgoKQmVsb3cgaXMgYSB0YWJsZSBmcm9tIHRoZSBbRG9ub2h1ZSwgZXQgYWwuXShodHRwczovL3d3dy5uYmVyLm9yZy9wYXBlcnMvdzIzNTEwLnBkZil7dGFyZ2V0PSJfYmxhbmsifSBwYXBlciB0aGF0IHNob3dzIHRoZSBkYXRhIHVzZWQgaW4gYm90aCBhbmFseXNlcywgd2hlcmUgREFXIHN0YW5kcyBmb3IgW0Rvbm9odWUsIGV0IGFsLl0oaHR0cHM6Ly93d3cubmJlci5vcmcvcGFwZXJzL3cyMzUxMC5wZGYpe3RhcmdldD0iX2JsYW5rIn0gYW5kIExNIHN0YW5kcyBmb3IgW011c3RhcmQgYW5kIExvdHRdKGh0dHBzOi8vY2hpY2Fnb3VuYm91bmQudWNoaWNhZ28uZWR1L2NnaS92aWV3Y29udGVudC5jZ2k/YXJ0aWNsZT0xMTUwJmNvbnRleHQ9bGF3X2FuZF9lY29ub21pY3Mpe3RhcmdldD0iX2JsYW5rIn0uCgoKYGBge3IsIGVjaG89RkFMU0UsIG91dC5oZWlnaHQgPSAnMTAwJScsIG91dC53aWR0aCA9ICcxMDAlJywgZmlnLmFsaWduPSdjZW50ZXInfQprbml0cjo6aW5jbHVkZV9ncmFwaGljcyhoZXJlKCJpbWciLCAiRG9ub2h1ZV9BcHBlbmRpeEoucG5nIikpCmBgYAoKV2Ugd2lsbCBiZSB1c2luZyBhIHN1YnNldCBvZiB0aGVzZSB2YXJpYWJsZXMsIHdoaWNoIGFyZSBoaWdobGlnaHRlZCBpbiBncmVlbjoKCgpgYGB7ciwgZWNobz1GQUxTRSwgb3V0LmhlaWdodCA9ICcxMDAlJywgb3V0LndpZHRoID0gJzEwMCUnLCBmaWcuYWxpZ249J2NlbnRlcid9CmtuaXRyOjppbmNsdWRlX2dyYXBoaWNzKGhlcmUoImltZyIsICJvdXJkYXRhLnBuZyIpKQpgYGAKCgojICoqRGF0YSBJbXBvcnQgYW5kIFdyYW5nbGluZyoqCioqKgoKU2VlIFt0aGlzIGNhc2Ugc3R1ZHldKGh0dHBzOi8vd3d3Lm9wZW5jYXNlc3R1ZGllcy5vcmcvb2NzLWJwLVJUQy13cmFuZ2xpbmcvKXt0YXJnZXQ9Il9ibGFuayJ9IGZvciBkZXRhaWxzIGFib3V0IGRhdGEgaW1wb3J0IGFuZCB3cmFuZ2xpbmcuIFdlIGluY2x1ZGUgZGF0YSBzb3VyY2VzIGhlcmUgZm9yIGNvbXBsZXRlbmVzcywgYnV0IHdpbGwgc3RhcnQgZnJvbSBwcm9jZXNzZWQgZGF0YSBpbiB0aGlzIGNhc2Ugc3R1ZHkuCgojIyAqKkRlbW9ncmFwaGljIGFuZCBwb3B1bGF0aW9uIGRhdGEqKgoqKioKClRvIG9idGFpbiBpbmZvcm1hdGlvbiBhYm91dCBhZ2UsIHNleCwgcmFjZSwgYW5kIG92ZXJhbGwgcG9wdWxhdGlvbiBzaXplIHdlIHdpbGwgdXNlIFVTIENlbnN1cyBCdXJlYXUgZGF0YSwgYXMgd2FzIGRvbmUgaW4gYm90aCBvZiB0aGUgYXJ0aWNsZXMuIFRoZSBjZW5zdXMgZGF0YSBpcyBhdmFpbGFibGUgZm9yIGRpZmZlcmVudCB0aW1lIHNwYW5zLiBIZXJlIGFyZSB0aGUgbGlua3MgZm9yIHRoZSB5ZWFycyB1c2VkIGluIG91ciBhbmFseXNpcy4gV2Ugd2lsbCB1c2UgZGF0YSBmcm9tIDE5NzcgdG8gMjAxMC4KCkRhdGEgICB8IExpbmsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKLS0tLS0tLS0tLSB8LS0tLS0tLS0tLS0tLQoqKnllYXJzIDE5NzcgdG8gMTk3OSoqICB8IFtsaW5rXShodHRwczovL3d3dzIuY2Vuc3VzLmdvdi9wcm9ncmFtcy1zdXJ2ZXlzL3BvcGVzdC90YWJsZXMvMTkwMC0xOTgwL3N0YXRlL2FzcmgvKSAgCioqeWVhcnMgMTk4MCB0byAxOTg5KiogIHwgW2xpbmtdKGh0dHBzOi8vd3d3Mi5jZW5zdXMuZ292L3Byb2dyYW1zLXN1cnZleXMvcG9wZXN0L3RhYmxlcy8xOTgwLTE5OTAvY291bnRpZXMvYXNyaC8pICogY291bnR5IGRhdGEgd2FzIHVzZWQgZm9yIHRoaXMgZGVjYWRlIHdoaWNoIGFsc28gaGFzIHN0YXRlIGluZm9ybWF0aW9uCioqeWVhcnMgMTk5MCB0byAxOTk5KiogIHwgW2xpbmtdKGh0dHBzOi8vd3d3Mi5jZW5zdXMuZ292L3Byb2dyYW1zLXN1cnZleXMvcG9wZXN0L3RhYmxlcy8xOTkwLTIwMDAvc3RhdGUvYXNyaC8pCioqeWVhcnMgMjAwMCB0byAyMDEwKiogIHwgW2xpbmtdKGh0dHBzOi8vd3d3LmNlbnN1cy5nb3YvZGF0YS9kYXRhc2V0cy90aW1lLXNlcmllcy9kZW1vL3BvcGVzdC9pbnRlcmNlbnNhbC0yMDAwLTIwMTAtc3RhdGUuaHRtbCkgPGJyPiBbdGVjaG5pY2FsIGRvY3VtZW50YXRpb25dKGh0dHBzOi8vd3d3Mi5jZW5zdXMuZ292L3Byb2dyYW1zLXN1cnZleXMvcG9wZXN0L3RlY2huaWNhbC1kb2N1bWVudGF0aW9uL2ZpbGUtbGF5b3V0cy8yMDAwLTIwMTAvaW50ZXJjZW5zYWwvc3RhdGUvc3QtZXN0MDBpbnQtYWxsZGF0YS5wZGYpe3RhcmdldD0iX2JsYW5rIn0KCiMjICoqU3RhdGUgRklQUyBjb2RlcyoqCioqKgoKVGhlICBkYXRhIHdhcyBkb3dubG9hZGVkIGZyb20gdGhlIFtVUyBDZW5zdXMgQnVyZWF1XShodHRwczovL3d3dy5jZW5zdXMuZ292L2dlb2dyYXBoaWVzL3JlZmVyZW5jZS1maWxlcy8yMDE0L2RlbW8vcG9wZXN0LzIwMTQtZ2VvY29kZXMtc3RhdGUuaHRtbCl7dGFyZ2V0PSJfYmxhbmsifS4KCiMjICoqUG9saWNlIHN0YWZmaW5nIGRhdGEqKgoqKioKClRoZSBmb2xsb3dpbmcgZGF0YSB3YXMgZG93bmxvYWRlZCBmcm9tIHRoZSBbRmVkZXJhbCBCdXJlYXUgb2YgSW52ZXN0aWdhdGlvbl0oaHR0cHM6Ly9jcmltZS1kYXRhLWV4cGxvcmVyLmZyLmNsb3VkLmdvdi9kb3dubG9hZHMtYW5kLWRvY3Mpe3RhcmdldD0iX2JsYW5rIn0uIAoKIyMgKipVbmVtcGxveW1lbnQgZGF0YSoqCioqKgoKVGhlIGZvbGxvd2luZyBkYXRhIHdhcyBkb3dubG9hZGVkIGZyb20gdGhlIFtVLlMuIEJ1cmVhdSBvZiBMYWJvciBTdGF0aXN0aWNzXShodHRwczovL2RhdGEuYmxzLmdvdi9jZ2ktYmluL2RzcnY/bGEpe3RhcmdldD0iX2JsYW5rIn0uIAoKIyMgKipQb3ZlcnR5IGRhdGEqKgoqKioKCkV4dHJhY3RlZCBmcm9tIFRhYmxlIDIxIGZyb20gW1VTIENlbnN1cyBCdXJlYXUgUG92ZXJ0eSBEYXRhIF0oaHR0cHM6Ly93d3cuY2Vuc3VzLmdvdi9kYXRhL3RhYmxlcy90aW1lLXNlcmllcy9kZW1vL2luY29tZS1wb3ZlcnR5L2hpc3RvcmljYWwtcG92ZXJ0eS1wZW9wbGUuaHRtbCl7dGFyZ2V0PSJfYmxhbmsifS4KCiMjICoqVmlvbGVudCBjcmltZSoqCioqKgoKVmlvbGVudCBjcmltZSBkYXRhIHdhcyBvYnRhaW5lZCBmcm9tIFtoZXJlXShodHRwczovL3d3dy51Y3JkYXRhdG9vbC5nb3YvU2VhcmNoL0NyaW1lL1N0YXRlL1N0YXRlYnlTdGF0ZS5jZm0pe3RhcmdldD0iX2JsYW5rIn0uIAoKIyMgKipSaWdodC10by1jYXJyeSBkYXRhKioKKioqCgpUaGlzIGRhdGEgaXMgZXh0cmFjdGVkIGZyb20gdGFibGUgaW4gW0Rvbm9odWUgcGFwZXJdKGh0dHBzOi8vd3d3Lm5iZXIub3JnL3BhcGVycy93MjM1MTAucGRmKXt0YXJnZXQ9Il9ibGFuayJ9LiAKCgoKKioqCgpIZXJlIGlzIHRoZSB0YWJsZSBmcm9tIHRoZSBbRG9ub2h1ZSBwYXBlcl0oaHR0cHM6Ly93d3cubmJlci5vcmcvcGFwZXJzL3cyMzUxMC5wZGYpe3RhcmdldD0iX2JsYW5rIn0gdGhhdCBjb21wYXJlcyB0aGUgZGF0YSB1c2VkIGluIHRoZSBhbmFseXNlczoKCgpgYGB7ciwgZWNobz1GQUxTRSwgb3V0LmhlaWdodCA9ICcxMDAlJywgb3V0LndpZHRoID0gJzEwMCUnLCBmaWcuYWxpZ249J2NlbnRlcid9CmtuaXRyOjppbmNsdWRlX2dyYXBoaWNzKGhlcmUoImltZyIsICJEb25vaHVlX1RhYmxlMl9lZGl0ZWQucG5nIikpCmBgYAoKIyMjIyMgW1tzb3VyY2VdXShodHRwczovL3d3dy5uYmVyLm9yZy9wYXBlcnMvdzIzNTEwLnBkZil7dGFyZ2V0PSJfYmxhbmsifQoKCiMjIyMjIyAqTUwgaXMgYWJicmV2aWF0ZWQgYXMgTE0gaW4gdGhlIHNvdXJjZSBhcnRpY2xlCgpXZSBjYW4gc2VlIHRoYXQgb25seSB0aGUgcGVyY2VudGFnZSBvZiBtYWxlcyB0aGF0IHdlcmUgZnJvbSBhZ2UgMTUtMzkgb2YgdGhlIHJhY2UgZ3JvdXBzIChibGFjaywgd2hpdGUsIGFuZCBvdGhlcikgd2VyZSB1c2VkIGluIHRoZSBEb25vaHVlIGFuYWx5c2lzLCB3aGVyZWFzIHRoZSBNdXN0YXJkIGFuZCBMb3R0IHBhcGVyLCBpbmRpdmlkdWFscyBmcm9tIGFnZXMgMTAgdG8gb3ZlciA2NSB3ZXJlIHVzZWQuCgpUaGUgZmluYWwgcHJvZHVjdHMgZnJvbSBvdXIgZGF0YSB3cmFuZ2xpbmcgd2hpY2ggaXMgZGVzY3JpYmVkIGluIFthbm90aGVyIGNhc2Ugc3R1ZHldKGh0dHBzOi8vd3d3Lm9wZW5jYXNlc3R1ZGllcy5vcmcvb2NzLWJwLVJUQy13cmFuZ2xpbmcvKXt0YXJnZXQ9Il9ibGFuayJ9IGFyZSB0d28gdGliYmxlcyBvZiBkYXRhIGVhY2ggd2l0aCB2YXJpYWJsZXMgc2VsZWN0ZWQgdG8gYmUgc2ltaWxhciB0byB0aG9zZSB1c2VkIGluIGVpdGhlciB0aGUgRG9ub2h1ZSBvciBNdXN0YXJkIGFuZCBMb3R0IGFuYWx5c2lzLiBUaGUgb3ZlcmFyY2hpbmcgaWRlYSBvZiB0aGlzIGNhc2Ugc3R1ZHkgaXMgdG8gY29tcGFyZSB0aGUgbW9kZWwgcmVzdWx0cyBmcm9tIHRoZXNlIHR3byBkYXRhc2V0cywgd2hpY2ggZGlmZmVyIG9ubHkgaW4gdGhlIGRlbW9ncmFwaGljIHZhcmlhYmxlcy4gVGhpcyBpcyBhbiBvdmVyc2ltcGxpZmljYXRpb24gb2YgdGhlIGFjdHVhbCBkaWZmZXJlbmNlcyBiZXR3ZWVuIHRoZSBkYXRhc2V0cyBhbmQgYXBwcm9hY2hlcyB0YWtlbiBieSBEb25vaHVlIGFuZCBMb3R0L011c3RhcmQsIGJ1dCB3aWxsIGJlIHVzZWZ1bCBmb3IgaWxsdXN0cmF0aW5nIHRoZSBpbXBhY3Qgb2YgbXVsdGljb2xsaW5lYXJpdHkgb24gb3VyIG1vZGVsaW5nIHJlc3VsdHMsIHdoaWNoIGlzIHRoZSBtYWluIG9iamVjdGl2ZSBvZiB0aGlzIGNhc2Ugc3R1ZHkuCgpJZiB5b3UgaGF2ZSB0cm91YmxlIGFjY2Vzc2luZyB0aGUgR2l0SHViIFJlcG9zaXRvcnksIHRoZSAiV3JhbmdsZWRfZGF0YS5yZGEiIGRhdGEgY2FuIGJlIGRvd25sb2FkZWQgZnJvbSBbaGVyZV0oaHR0cHM6Ly9naXRodWIuY29tL29wZW5jYXNlc3R1ZGllcy9vY3MtYnAtUlRDLWFuYWx5c2lzL2Jsb2IvbWFzdGVyL2RhdGEvV3JhbmdsZWRfZGF0YS5yZGEpLgoKV2Ugd2lsbCBsb2FkIHRoaXMgZGF0YSBub3cuIAoKSW4gb3VyIGNhc2UsIHdlIGRvd25sb2FkZWQgdGhpcyBkYXRhIGFuZCBwdXQgaXQgd2l0aGluIGEgIndyYW5nbGVkIiBzdWJkaXJlY3Rvcnkgb2YgdGhlICJkYXRhIiBkaXJlY3RvcnkgZm9yIG91ciBwcm9qZWN0LiBJZiB5b3UgdXNlIGFuIFJTdHVkaW8gcHJvamVjdCwgdGhlbiB5b3UgY2FuIHVzZSB0aGUgYGhlcmUoKWAgZnVuY3Rpb24gb2YgdGhlIGBoZXJlYCBwYWNrYWdlIHRvIG1ha2UgdGhlIHBhdGggZm9yIGxvYWRpbmcgdGhpcyBkYXRhIHNpbXBsZXIuIFRoZSBgaGVyZWAgcGFja2FnZSBhdXRvbWF0aWNhbGx5IHN0YXJ0cyBsb29raW5nIGZvciBmaWxlcyBiYXNlZCBvbiB3aGVyZSB5b3UgaGF2ZSBhIGAuUnByb2pgIGZpbGUgd2hpY2ggaXMgY3JlYXRlZCB3aGVuIHlvdSBzdGFydCBhIG5ldyBSU3R1ZGlvIHByb2plY3QuIFdlIGNhbiBzcGVjaWZ5IHRoYXQgd2Ugd2FudCB0byBsb29rIGZvciB0aGUgZmlsZSB3aXRoaW4gdGhlICJ3cmFuZ2xlZCIgc3ViZGlyZWN0b3J5IG9mIHRoZSAiZGF0YSIgZGlyZWN0b3J5IHdpdGhpbiBhIGRpcmVjdG9yeSB3aGVyZSBvdXIgYC5ScHJvamAgZmlsZSBpcyBsb2NhdGVkIGJ5IHNlcGFyYXRpbmcgdGhlIG5hbWUgb2YgdGhlICJkYXRhIiBkaXJlY3RvcnkgYW5kICJ3cmFuZ2xlZCIgYW5kIHRoZSBmaWxlIG5hbWUgdXNpbmcgY29tbWFzLgoKKioqCjxkZXRhaWxzPiA8c3VtbWFyeT4gQ2xpY2sgaGVyZSB0byBzZWUgbW9yZSBhYm91dCBjcmVhdGluZyBuZXcgcHJvamVjdHMgaW4gUlN0dWRpby4gPC9zdW1tYXJ5PgoKWW91IGNhbiBjcmVhdGUgYSBwcm9qZWN0IGJ5IGdvaW5nIHRvIHRoZSBGaWxlIG1lbnUgb2YgUlN0dWRpbyBsaWtlIHNvOgoKCmBgYHtyLCBlY2hvID0gRkFMU0UsIG91dC53aWR0aD0iNjAlIn0Ka25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MoaGVyZTo6aGVyZSgiaW1nIiwgIk5ld19wcm9qZWN0LnBuZyIpKQpgYGAKCllvdSBjYW4gYWxzbyBkbyBzbyBieSBjbGlja2luZyB0aGUgcHJvamVjdCBidXR0b246CgpgYGB7ciwgZWNobyA9IEZBTFNFLCBvdXQud2lkdGg9IjYwJSJ9CmtuaXRyOjppbmNsdWRlX2dyYXBoaWNzKGhlcmU6OmhlcmUoImltZyIsICJwcm9qZWN0X2J1dHRvbi5wbmciKSkKYGBgCgpTZWUgW2hlcmVdKGh0dHBzOi8vc3VwcG9ydC5yc3R1ZGlvLmNvbS9oYy9lbi11cy9hcnRpY2xlcy8yMDA1MjYyMDctVXNpbmctUHJvamVjdHMpIHRvIGxlYXJuIG1vcmUgYWJvdXQgdXNpbmcgUlN0dWRpbyBwcm9qZWN0cy4gCgo8L2RldGFpbHM+CgoqKioKCmBgYHtyfQpsb2FkKGZpbGUgPSBoZXJlOjpoZXJlKCJkYXRhIiwgIndyYW5nbGVkIiwgIldyYW5nbGVkX2RhdGEucmRhIikpCmBgYAoKWW91IG1heSBhbHNvIGRvd25sb2FkIHRoZSBkYXRhIGZpbGVzIHdpdGggdGhlIGBPQ1NkYXRhYCBwYWNrYWdlLiBJZiB5b3UgcHJlZmVyIHRoaXMgbWV0aG9kLCB1c2UgdGhlIGZvbGxvd2luZyBjb2RlIGNodW5rIHRvIGRvd25sb2FkIGFuZCBsb2FkIHRoZSBkYXRhOgoKYGBge3IsIGV2YWw9RkFMU0V9CiMgaW5zdGFsbC5wYWNrYWdlcygiT0NTZGF0YSIpCmxpYnJhcnkoT0NTZGF0YSkKd3JhbmdsZWRfcmRhKCJvY3MtYnAtUlRDLWFuYWx5c2lzIiwgb3V0cGF0aCA9IGdldHdkKCkpCiMgVGhpcyB3aWxsIHNhdmUgdGhlIHdyYW5nbGVkIGRhdGEgZmlsZXMgaW4gYSAiT0NTZGF0YS9kYXRhL3dyYW5nbGVkLyIgCiMgc3ViLWZvbGRlciBpbiB5b3VyIGN1cnJlbnQgd29ya2luZyBkaXJlY3RvcnkKbG9hZChmaWxlID0gaGVyZTo6aGVyZSgiT0NTZGF0YSIsICJkYXRhIiwgIndyYW5nbGVkIiwgIldyYW5nbGVkX2RhdGEucmRhIikpCiMgdGhpcyB3aWxsIGxvYWQgdGhlIGRhdGEgaW50byBSCmBgYAoKV2Ugd2lsbCBjaGVjayB0aGUgZGltZW5zaW9ucyBvZiBlYWNoIHRpYmJsZSB1c2luZyB0aGUgYmFzZSBgZGltKClgIGZ1bmN0aW9uOgpgYGB7cn0KZGltKExPVFRfREYpCmRpbShET05PSFVFX0RGKQpgYGAKCkFzIGV4cGVjdGVkIHRoZSBgTG90dF9ERmAgaXMgMzAgY29sdW1ucyBsYXJnZXIsIGR1ZSB0byB0aGUgMzAgYWRkaXRpb25hbCBkZW1vZ3JhcGhpYyB2YXJpYWJsZXMuIFdlIGNhbiBjaGVjayB0aG9zZSBub3cgYXMgd2VsbC4KCmBgYHtyfQpMT1RUX0RGICU+JQogIGNvbG5hbWVzKCkKCkRPTk9IVUVfREYgJT4lCiAgY29sbmFtZXMoKQpgYGAKCkxhc3RseSwgd2Ugd2lsbCBjaGVjayB0aGF0IHRoZSBgWUVBUmAgdmFsdWVzIGFyZSB0aGUgc2FtZSwgaS5lLiwgdGhhdCB0aGUgdHdvIHRpYmJsZXMgY29udGFpbiBkYXRhIGZyb20gdGhlIHNhbWUgc2V0IG9mIHllYXJzLiBXZSBjYW4gdXNlIHRoZSBgc2V0ZXF1YWwoKWAgZnVuY3Rpb24gb2YgdGhlIGBkcGx5cmAgcGFja2FnZSB0byBzZWUgaWYgdGhlIHZhbHVlcyBhcmUgdGhlIHNhbWUuIAoKYGBge3J9CnNldGVxdWFsKAogIERPTk9IVUVfREYgJT4lIGRpc3RpbmN0KFlFQVIpLAogIExPVFRfREYgJT4lIGRpc3RpbmN0KFlFQVIpCikKYGBgCgoKIyAqKkRhdGEgRXhwbG9yYXRpb24qKgoqKioKCkxldCdzIGRvIHNvbWUgcXVpY2sgdmlzdWFsaXphdGlvbnMgdG8gZ2V0IGEgc2Vuc2Ugb2Ygb3VyIG91dGNvbWUgb2YgaW50ZXJlc3QsIHRoZSB2aW9sZW50IGNyaW1lIGRhdGEuIAogCkZpcnN0IHdlIHdpbGwgcGxvdCB0aGUgcmF0ZSBvZiB2aW9sZW50IGNyaW1lIG92ZXIgdGltZSB0byBnZXQgYSBzZW5zZSBvZiB0aGUgZ2VuZXJhbCB0cmVuZC4KClRvIGRvIHNvIHdlIG5lZWQgdG8gc3VtbWFyaXplIHRoZSBkYXRhIGZvciBlYWNoIHllYXIgYWNyb3NzIGFsbCBvZiB0aGUgc3RhdGVzLiAKVGh1cyB3ZSB3aWxsIHVzZSB0aGUgYGdyb3VwX2J5KClgIGZ1bmN0aW9uIGFuZCB0aGUgYHN1bW1hcml6ZSgpYCBmdW5jdGlvbnMgdG8gY2FsY3VsYXRlIGFuIG92ZXJhbGwgbnVtYmVyIG9mIHZpb2xlbnQgY3JpbWVzIHJlbGF0aXZlIHRvIHRoZSB0b3RhbCBwb3B1bGF0aW9uIGZvciBlYWNoIHllYXIuIEluIGZhY3QsIHdlIHdpbGwgY2FsY3VsYXRlIHRoZSBsb2cgb2YgdGhlIG51bWJlciBvZiB2aW9sZW50IGNyaW1lcyBwZXIgMTAwLDAwMCBpbmRpdmlkdWFscywgYXMgdGhpcyB3aWxsIG1ha2UgaXQgZWFzaWVyIHRvIHNlZSB0aGUgdHJlbmQgaW4gdGhlIGRhdGEuIEluIGFkZGl0aW9uLCBieSBsb29raW5nIGF0IHRoZSByYXRlIHBlciAxMDAsMDAwIGluZGl2aWR1YWxzLCB3ZSBjYW4gbW9yZSBkaXJlY3RseSBjb21wYXJlIGNyaW1lIHN0YXRpc3RpY3MgYWNyb3NzIGdyb3VwcyBsaWtlIHN0YXRlcyB3aXRoIGRpZmZlcmVudCBwb3B1bGF0aW9ucywgYXMgeW91IHdpbGwgc2VlIGJlbG93LgoKVGhlbiB3ZSB3aWxsIHVzZSB0aGUgYGdncGxvdDJgIHBhY2thZ2UgdG8gcGxvdCB0aGUgZGF0YS4gVGhlIGZpcnN0IHN0ZXAgaW4gY3JlYXRpbmcgYSBwbG90IHdpdGggdGhpcyBwYWNrYWdlIGlzIHRvIHVzZSB0aGUgYGdncGxvdCgpYCBmdW5jdGlvbiBhbmQgdGhlIGBhZXMoKWAgYXJndW1lbnQgdG8gc3BlY2lmeSB3aGF0IGRhdGEgc2hvdWxkIGJlIHBsb3R0ZWQgb24gdGhlIHgtYXhpcyBhbmQgd2hhdCBkYXRhIHNob3VsZCBiZSBwbG90dGVkIGluIG9uIHRoZSB5LWF4aXMuIFRoZW4gd2Ugc2VsZWN0IHdoYXQgdHlwZSBvZiBwbG90IHdlIHdvdWxkIGxpa2UgdG8gbWFrZSB1c2luZyBvbmUgb2YgdGhlIGBnZW9tXyooKWAgZnVuY3Rpb25zLiBQbGVhc2Ugc2VlIFt0aGlzIGNhc2Ugc3R1ZHldKGh0dHBzOi8vb3BlbmNhc2VzdHVkaWVzLmdpdGh1Yi5pby9vY3MtYnAtY28yLWVtaXNzaW9ucy8pe3RhcmdldD0iX2JsYW5rIn0gIGZvciBtb3JlIGRldGFpbHMuCgpXZSBjYW4gdXNlIHRoZSBgc2NhbGVfeF9jb250aW51b3VzKClgIGFuZCBgc2NhbGVfeV9jb250aW51b3VzKClgIGZ1bmN0aW9ucyB0byBtb2RpZnkgdGhlIGF4aXMgdGljayBtYXJrcyBhbmQgdGhlaXIgbGFiZWxzLgoKVGhlIGBsYWJzKClgIGZ1bmN0aW9uIGNhbiBiZSB1c2VkIHRvIGFkZCBsYWJlbHMgdG8gdGhlIHBsb3QsIHdoaWxlIHRoZSBgdGhlbWUoKWAgZnVuY3Rpb24gYWxsb3dzIGZvciBtYW5pcHVsYXRpb24gb2YgdGhlIGRldGFpbHMgb2YgdGhlIGxhYmVscywgbGlrZSBzaXplIGFuZCBhbmdsZS4gCgpBbGwgb2YgdGhlc2UgZnVuY3Rpb25zIGFyZSBwYXJ0IG9mIHRoZSBgZ2dwbG90MmAgcGFja2FnZS4KIApgYGB7cn0KCkRPTk9IVUVfREYgJT4lCiAgZ3JvdXBfYnkoWUVBUikgJT4lCiAgc3VtbWFyaXplKAogICAgVmlvbF9jcmltZV9jb3VudCA9IHN1bShWaW9sX2NyaW1lX2NvdW50KSwKICAgIFBvcHVsYXRpb24gPSBzdW0oUG9wdWxhdGlvbiksCiAgICAuZ3JvdXBzID0gImRyb3AiCiAgKSAlPiUKICBtdXRhdGUoVmlvbF9jcmltZV9yYXRlXzEwMGtfbG9nID0gbG9nKChWaW9sX2NyaW1lX2NvdW50ICogMTAwMDAwKSAvIFBvcHVsYXRpb24pKSAlPiUKICBnZ3Bsb3QoYWVzKHggPSBZRUFSLCB5ID0gVmlvbF9jcmltZV9yYXRlXzEwMGtfbG9nKSkgKwogIGdlb21fbGluZSgpICsKICBzY2FsZV94X2NvbnRpbnVvdXMoCiAgICBicmVha3MgPSBzZXEoMTk4MCwgMjAxMCwgYnkgPSAxKSwKICAgIGxpbWl0cyA9IGMoMTk4MCwgMjAxMCksCiAgICBsYWJlbHMgPSBjKHNlcSgxOTgwLCAyMDEwLCBieSA9IDEpKQogICkgKwogIHNjYWxlX3lfY29udGludW91cygKICAgIGJyZWFrcyA9IHNlcSg1Ljc1LCA2Ljc1LCBieSA9IDAuMjUpLAogICAgbGltaXRzID0gYyg1Ljc1LCA2Ljc1KQogICkgKwogIGxhYnMoCiAgICB0aXRsZSA9ICJDcmltZSByYXRlcyBmbHVjdHVhdGUgb3ZlciB0aW1lIiwKICAgIHggPSAiWWVhciIsCiAgICB5ID0gImxuKHZpb2xlbnQgY3JpbWVzIHBlciAxMDAsMDAwIHBlb3BsZSkiCiAgKSArCiAgdGhlbWVfbWluaW1hbCgpICsKICB0aGVtZShheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChhbmdsZSA9IDkwKSkKYGBgCkludGVyZXN0aW5nISBJdCBhcHBlYXJzIHRoYXQgdGhlcmUgd2FzIGFuIG92ZXJhbGwgbmF0aW9uYWwgcGVhayBpbiB2aW9sZW50IGNyaW1lIGluIHRoZSBlYXJseSAxOTkwcyB0aGF0IGhhcyBzaW5jZSB0aGVuIGRlY2xpbmVkLiAgCgoKTm93IGxldCdzIHRha2UgYSBsb29rIGF0IGVhY2ggc3RhdGUuCgpXZSB3aWxsIHVzZSB0aGUgYGdncmVwZWxgIHBhY2thZ2UgdG8gYWRkIHRleHQgdG8gdGhlIHBsb3QgdXNpbmcgdGhlIGBnZW9tX3RleHRfcmVwZWwoKWAgZnVuY3Rpb24uIFRoaXMgaXMgZXNwZWNpYWxseSB1c2VmdWwgd2hlbiB0aGVyZSBpcyBhIGxvdCBvZiB0ZXh0LCBhcyB0aGlzIGZ1bmN0aW9uIHJlZHVjZXMgdGhlIG92ZXJsYXAgb2YgdGV4dCBsYWJlbHMuIEFnYWluIHNlZSBbdGhpcyBjYXNlIHN0dWR5XShodHRwczovL29wZW5jYXNlc3R1ZGllcy5naXRodWIuaW8vb2NzLWJwLWNvMi1lbWlzc2lvbnMvKXt0YXJnZXQ9Il9ibGFuayJ9ICBmb3IgbW9yZSBkZXRhaWxzIG9uIGhvdyB0byBhZGQgbGFiZWxzIHRvIGVsZW1lbnRzIG9mIHBsb3RzLgogCmBgYHtyfQoKRE9OT0hVRV9ERiAlPiUKICBtdXRhdGUoVmlvbF9jcmltZV9yYXRlXzEwMGtfbG9nID0gbG9nKChWaW9sX2NyaW1lX2NvdW50ICogMTAwMDAwKSAvIFBvcHVsYXRpb24pKSAlPiUKICBnZ3Bsb3QoYWVzKHggPSBZRUFSLCB5ID0gVmlvbF9jcmltZV9yYXRlXzEwMGtfbG9nLCBjb2xvciA9IFNUQVRFKSkgKwogIGdlb21fcG9pbnQoc2l6ZSA9IDAuNSkgKwogIGdlb21fbGluZShhZXMoZ3JvdXAgPSBTVEFURSksCiAgICBzaXplID0gMC41LAogICAgc2hvdy5sZWdlbmQgPSBGQUxTRQogICkgKwogIGdlb21fdGV4dF9yZXBlbCgKICAgIGRhdGEgPSBET05PSFVFX0RGICU+JQogICAgICBtdXRhdGUoVmlvbF9jcmltZV9yYXRlXzEwMGtfbG9nID0gbG9nKChWaW9sX2NyaW1lX2NvdW50ICogMTAwMDAwKSAvIFBvcHVsYXRpb24pKSAlPiUKICAgICAgZmlsdGVyKFlFQVIgPT0gbGFzdChZRUFSKSksCiAgICBhZXMoCiAgICAgIGxhYmVsID0gU1RBVEUsCiAgICAgIHggPSBZRUFSLAogICAgICB5ID0gVmlvbF9jcmltZV9yYXRlXzEwMGtfbG9nCiAgICApLAogICAgc2l6ZSA9IDMsCiAgICBhbHBoYSA9IDEsCiAgICBudWRnZV94ID0gMTAsCiAgICBkaXJlY3Rpb24gPSAieSIsCiAgICBoanVzdCA9IDEsCiAgICB2anVzdCA9IDEsCiAgICBzZWdtZW50LnNpemUgPSAwLjI1LAogICAgc2VnbWVudC5hbHBoYSA9IDAuMjUsCiAgICBmb3JjZSA9IDEsCiAgICBtYXguaXRlciA9IDk5OTkKICApICsKICBndWlkZXMoY29sb3IgPSBGQUxTRSkgKwogIHNjYWxlX3hfY29udGludW91cygKICAgIGJyZWFrcyA9IHNlcSgxOTgwLCAyMDE1LCBieSA9IDEpLAogICAgbGltaXRzID0gYygxOTgwLCAyMDE1KSwKICAgIGxhYmVscyA9IGMoc2VxKDE5ODAsIDIwMTAsIGJ5ID0gMSksIHJlcCgiIiwgNSkpCiAgKSArCiAgc2NhbGVfeV9jb250aW51b3VzKAogICAgYnJlYWtzID0gc2VxKDMuNSwgOC41LCBieSA9IDAuNSksCiAgICBsaW1pdHMgPSBjKDMuNSwgOC41KQogICkgKwogIGxhYnMoCiAgICB0aXRsZSA9ICJTdGF0ZXMgaGF2ZSBkaWZmZXJlbnQgbGV2ZWxzIG9mIGNyaW1lIiwKICAgIHggPSAiWWVhciIsCiAgICB5ID0gImxuKHZpb2xlbnQgY3JpbWVzIHBlciAxMDAsMDAwIHBlb3BsZSkiCiAgKSArCiAgdGhlbWVfbWluaW1hbCgpICsKICB0aGVtZShheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChhbmdsZSA9IDkwKSkKYGBgCgpJdCBsb29rcyBsaWtlIHRoZSBjcmltZSByYXRlcyB2YXJ5IHF1aXRlIGEgYml0IGZyb20gb25lIHN0YXRlIHRvIGFub3RoZXIuIFNvbWUgc3RhdGVzIHNob3cgaW5jcmVhc2VkIGNyaW1lIG92ZXIgdGltZSB3aGlsZSBvdGhlcnMgc2hvdyBkZWNyZWFzZWQgY3JpbWUuIAoKCk5vdyBsZXQncyB0YWtlIGEgY2xvc2VyIGxvb2sgYXQgc29tZSBvZiBvdXIgb3RoZXIgdmFyaWFibGVzLgoKQXMgd2UgZG8gdGhpcywgd2UgbWF5IHN0YXJ0IHRvIHJ1biBpbnRvIGlzc3VlcyB3aXRoIG1pc3NpbmcgZGF0YSBmb3Igc29tZSBvZiBvdXIgdmFyaWFibGVzLCBhbmQgaXQgaXMgYWx3YXlzIGdvb2QgdG8gYmUgYXdhcmUgb2YgbWlzc2luZyBkYXRhIGluIGFueSBkYXRhIGFuYWx5c2lzLiBXZSBjYW4gdXNlIHRoZSBgdmlzX21pc3MoKWAgZnVuY3Rpb24gb2YgdGhlIGBuYW5pYXJgIHBhY2thZ2UgdG8gY29uZmlybSB0aGF0IHRoZXJlIGFyZSBubyBtaXNzaW5nIHZhbHVlcy4KCmBgYHtyfQpET05PSFVFX0RGICU+JQogIG5hbmlhcjo6dmlzX21pc3MoKQpgYGAKCkxvb2tzIGxpa2Ugbm8gbWlzc2luZyBkYXRhISAKCmBgYHtyfQpMT1RUX0RGICU+JQogIG5hbmlhcjo6dmlzX21pc3MoKQpgYGAKU2FtZSBmb3IgdGhlIGBMT1RUX0RGYC4KCldlIGNhbiB1c2UgdGhlIGBza2ltKClgIG9mIHRoZSBgc2tpbXJgIHBhY2thZ2UgdG8gZ2V0IGEgYmV0dGVyIHNlbnNlIG9mIHRoZSBkYXRhLiBUaGlzIGFsc28gc2hvd3MgbWlzc2luZ25lc3MsIGFzIHdlbGwgYXMgc3RhbmRhcmQgZGV2aWF0aW9ucywgc3ByZWFkLCBhbmQgbWVhbnMgZm9yIG91ciBkYXRhLiBBbHNvIG5vdGljZSB0aGF0IHRoZXJlIGlzIGEgc21hbGwgaGlzdG9ncmFtIG9mIGVhY2ggdmFyaWFibGUgaW4gdGhlIGZpbmFsIGNvbHVtbi4KCmBgYHtyLCBldmFsPUZBTFNFfQpza2ltcjo6c2tpbShET05PSFVFX0RGKQpgYGAKCmBgYHtyLCBlY2hvPUZBTFNFLCBvdXQuaGVpZ2h0ID0gJzEwMCUnLCBvdXQud2lkdGggPSAnMTAwJScsIGZpZy5hbGlnbj0nY2VudGVyJ30Ka25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MoaGVyZSgiaW1nIiwgInNraW0xLnBuZyIpKQprbml0cjo6aW5jbHVkZV9ncmFwaGljcyhoZXJlKCJpbWciLCAic2tpbTIucG5nIikpCmBgYAoKYGBge3IsIGV2YWw9RkFMU0V9CnNraW1yOjpza2ltKExPVFRfREYpCmBgYAoKYGBge3IsIGVjaG89RkFMU0UsIG91dC5oZWlnaHQgPSAnMTAwJScsIG91dC53aWR0aCA9ICcxMDAlJywgZmlnLmFsaWduPSdjZW50ZXInfQprbml0cjo6aW5jbHVkZV9ncmFwaGljcyhoZXJlKCJpbWciLCAic2tpbTMucG5nIikpCmtuaXRyOjppbmNsdWRlX2dyYXBoaWNzKGhlcmUoImltZyIsICJza2ltNC5wbmciKSkKa25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MoaGVyZSgiaW1nIiwgInNraW01LnBuZyIpKQpgYGAKCldlIGNhbiBzZWUgZnJvbSB0aGlzIGZ1bmN0aW9uIHRoYXQgd2UgaGF2ZSB0aGUgbnVtYmVyIG9mIHZhcmlhYmxlcyBvZiB0aGUgY2xhc3MgdHlwZXMgdGhhdCB3ZSBleHBlY3QgZm9yIGVhY2ggdGliYmxlLiBXZSBjYW4gYWxzbyBzZWUgdGhhdCB0aGUgbWVhbnMgb2YgdGhlIHZhcmlhYmxlcyB0aGF0IHNob3VsZCBiZSB0aGUgc2FtZSBmb3IgZWFjaCB0aWJibGUgYXJlIGluIGZhY3QgdGhlIHNhbWUuIApXZSBjYW4gYWxzbyB0ZWxsIHRoYXQgdGhlIHZhbHVlcyBmb3IgdGhlIHZhcmlhYmxlcyBhcmUgaW4gZ2VuZXJhbCB3aGF0IHdlIHdvdWxkIGV4cGVjdC4KCiMgKipEYXRhIEFuYWx5c2lzKioKKioqCgojIyAqKlBhbmVsIEFuYWx5c2lzKioKKioqCgpJbiB0aGVzZSBkYXRhc2V0cywgd2UgaGF2ZSB3aGF0IGlzIGNhbGxlZCBbcGFuZWwgZGF0YV0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvUGFuZWxfZGF0YSl7dGFyZ2V0PSJfYmxhbmsifSwgYSBzcGVjaWFsIHR5cGUgb2YgbG9uZ2l0dWRpbmFsIGRhdGEuIExvbmdpdHVkaW5hbCBkYXRhIGFyZSBkYXRhIG1lYXN1cmVtZW50cyB0YWtlbiBvdmVyIHRpbWUuIFBhbmVsIGRhdGEgYXJlIGRhdGEgcmVwZWF0ZWRseSBtZWFzdXJlZCBmb3IgbXVsdGlwbGUgcGFuZWwgbWVtYmVycyBvciBpbmRpdmlkdWFscyBvdmVyIHRpbWUuIFRoaXMgaXMgaW4gY29udHJhc3Qgd2l0aCB0aW1lIHNlcmllcyBkYXRhLCB3aGljaCBtZWFzdXJlcyBvbmUgaW5kaXZpZHVhbCBvdmVyIHRpbWUgYW5kIGNyb3NzIHNlY3Rpb25hbCBkYXRhLCB3aGljaCBtZWFzdXJlcyBtdWx0aXBsZSBpbmRpdmlkdWFscyBhdCBvbmUgcG9pbnQgaW4gdGltZS4gIEluIG90aGVyIHdvcmRzLCBwYW5lbCBkYXRhIGlzIGEgY29tYmluYXRpb24gb2YgYm90aCwgd2l0aCBtZWFzdXJlbWVudHMgZm9yIG11bHRpcGxlIGluZGl2aWR1YWxzL3VuaXRzIG9mIG9ic2VydmF0aW9uIG92ZXIgbXVsdGlwbGUgdGltZSBwZXJpb2RzLiAgSW4gb3VyIGNhc2UsIHdlIGhhdmUgbWVhc3VyZW1lbnRzIG9mIHZpb2xlbnQgY3JpbWUgYW5kIG90aGVyIHZhcmlhYmxlcyBmb3IgZWFjaCBzdGF0ZSBvdmVyIG1hbnkgeWVhcnMuIFRoZXJlZm9yZSB3ZSBhcmUgdXNpbmcgbWVhc3VyZW1lbnRzIGFib3V0IHRoZSBzYW1lIHN0YXRlcyBvdmVyIHRpbWUuICAKCkluIGEgcGFuZWwgYW5hbHlzaXMgdGhlcmUgYXJlICROJCBpbmRpdmlkdWFsIHBhbmVsIG1lbWJlcnMgYW5kICRUJCB0aW1lIHBvaW50cy4gIAoKVGhlcmUgYXJlIHR3byB0eXBlcyBvZiBwYW5lbHM6ICAKMS4gKipCYWxhbmNlZCoqIC0gQXQgZWFjaCB0aW1lIHBvaW50ICgkVCQpLCB0aGVyZSBhcmUgZGF0YSBwb2ludHMgZm9yIGVhY2ggaW5kaXZpZHVhbCgkTiQpLiAKClRpbWUgUG9pbnRzICgkVCQpICB8IEluZGl2aWR1YWxzICgkTiQpICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAotLS0tLS0tLS0tIHwtLS0tLS0tLS0tLS0tCjE5NzcgIHwgTmV2YWRhIAoxOTc3ICB8IEFsYWJhbWEKMTk3NyAgfCBLYW5zYXMKMTk3OCB8IE5ldmFkYQoxOTc4IHwgQWxhYmFtYQoxOTc4ICB8IEthbnNhcwoxOTc5IHwgTmV2YWRhCjE5NzkgfCBBbGFiYW1hCjE5NzkgfCBLYW5zYXMKCjIuICoqVW5iYWxhbmNlZCoqIC0gVGhlcmUgbWF5IGJlIGRhdGEgcG9pbnRzIG1pc3NpbmcgZm9yIHNvbWUgaW5kaXZpZHVhbHMgKCROJCkgYXQgc29tZSB0aW1lIHBvaW50cyAoJFQkKS4KClRpbWUgUG9pbnRzICgkVCQpICB8IEluZGl2aWR1YWxzICgkTiQpICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAotLS0tLS0tLS0tIHwtLS0tLS0tLS0tLS0tCjE5NzcgIHwgTmV2YWRhIAoxOTc3ICB8IEFsYWJhbWEKMTk3OCB8IE5ldmFkYQoxOTc4IHwgQWxhYmFtYQoxOTc5IHwgTmV2YWRhCjE5NzkgfCBBbGFiYW1hCjE5NzkgfCBLYW5zYXMKCgpPdmVyYWxsIGluIGEgYmFsYW5jZWQgcGFuZWwsIHdlIGhhdmUgJG4kIG9ic2VydmF0aW9ucywgd2hlcmUgJG4gPSBOKlQkLiAgCgpJbiBhbiB1bmJhbGFuY2VkIHBhbmVsLCB0aGUgbnVtYmVyIG9mIG9ic2VydmF0aW9ucyBpcyBsZXNzIHRoYW4gJE4qVCQuCgoKSW4gb3VyIGNhc2Ugd2UgaGF2ZTogIAokTiQgPSA0NSBzdGF0ZXMgKGluIHRoZSBkYXRhIHdyYW5nbGluZyBwcm9jZXNzIHdlIHJlbW92ZWQgdGhvc2Ugd2hvIGhhZCBhZG9wdGVkIGFuIFJUQyBsYXcgYmVmb3JlIDE5ODApICAKJFQkID0gIDMxIHllYXJzICgxOTgwIC0gMjAxMCkgIAoKSW4gZXZlcnkgeWVhciB3ZSBoYXZlIG1lYXN1cmVtZW50cyBmb3IgZWFjaCBzdGF0ZSAoYXMgd2UganVzdCBzYXcgYWJvdmUpLCB0aHVzIG91ciBwYW5lbCBpcyBiYWxhbmNlZC4KClNvLCBvdXIgdG90YWwgb2JzZXJ2YXRpb25zICRuID0gNDUqMzEkLCB0aHVzICRuJCA9IGByIDQ1KjMxYC4gIAoKCgpXZSB3aWxsIGJlIHBlcmZvcm1pbmcgYSAqKnBhbmVsIGxpbmVhciByZWdyZXNzaW9uIG1vZGVsIGFuYWx5c2lzKiouIAoKCkluIHN1Y2ggYW4gYW5hbHlzaXMgd2Ugd2lsbCBtb2RlbCBvdXIgZGF0YSBhY2NvcmRpbmcgdG8gdGhpcyBnZW5lcmljIG1vZGVsOgoKJCRZX3tpdH09zrJfezB9K86yX3sxfVhfezFpdH0rLi4uK86yX3tLfVhfe0tpdH0rZV97aXR9JCQKCldoZXJlICRpJCBpcyB0aGUgaW5kaXZpZHVhbCBkaW1lbnNpb24gKGluIG91ciBjYXNlIGluZGl2aWR1YWwgc3RhdGVzKSBhbmQgJHQkIGlzIHRoZSB0aW1lIGRpbWVuc2lvbi4KClNvbWUgZXhwbGFuYXRvcnkvaW5kZXBlbmRlbnQgdmFyaWFibGVzIG9yIHJlZ3Jlc3NvcnMgJFhfe2l0fSQgd2lsbCB2YXJ5IGFjcm9zcyBpbmRpdmlkdWFscyBhbmQgdGltZSwgd2hpbGUgb3RoZXJzIHdpbGwgYmUgZml4ZWQgYWNyb3NzIHRoZSB0aW1lIG9mIHRoZSBzdHVkeSAob3IgZG9uJ3QgY2hhbmdlIG92ZXIgdGltZSksIHdoaWxlIG90aGVycyBzdGlsbCB3aWxsIGJlIGZpeGVkIGFjcm9zcyBpbmRpdmlkdWFscyBidXQgdmFyeSBhY3Jvc3MgdGltZSBwZXJpb2RzLgoKClRoZXJlIGFyZSB0aHJlZSBnZW5lcmFsIHN1Yi10eXBlcyBvZiBbcGFuZWwgcmVncmVzc2lvbiBhbmFseXNpc10oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvUGFuZWxfYW5hbHlzaXMpe3RhcmdldD0iX2JsYW5rIn0uCgpPdmVyYWxsLCB0aGV5IGFzc3VtZSB0aGF0IHRoZSBkaWZmZXJlbnQgaW5kaXZpZHVhbHMgYXJlIGluZGVwZW5kZW50LCBob3dldmVyIHRoZSBzYW1lIGRhdGEgZm9yIHRoZSBzYW1lIGluZGl2aWR1YWwgbWF5IGJlIGNvcnJlbGF0ZWQgYWNyb3NzIHRpbWUuCgpUaGUgbWFpbiBkaWZmZXJlbmNlIGJldHdlZW4gdGhlIHRocmVlIHN1Yi10eXBlcyBhcmUgdGhlIGFzc3VtcHRpb25zIGFib3V0IHVub2JzZXJ2ZWQgZGlmZmVyZW5jZXMgYmV0d2VlbiBpbmRpdmlkdWFscy4KCklmIHlvdSBhcmUgZmFtaWxpYXIgd2l0aCBmaXhlZCBhbmQgcmFuZG9tIGVmZmVjdHMgaW4gdGhlIGNvbnRleHQgaW4gdGhlIHN0YXRpc3RpY2FsIGxpdGVyYXR1cmUgdGhlIGZvbGxvd2luZyBlY29ub21ldHJpYyBkZWZpbml0aW9ucyB3aWxsIHNvdW5kIGEgYml0IGRpZmZlcmVudC4KCkZyb20gdGhlIGBwbG1gIHBhY2thZ2UgW3ZpZ25ldHRlXShodHRwczovL2NyYW4uci1wcm9qZWN0Lm9yZy93ZWIvcGFja2FnZXMvcGxtL3ZpZ25ldHRlcy9wbG1QYWNrYWdlLmh0bWwpOgoKPiBJbiB0aGUgbWl4ZWQgbW9kZWxzIGxpdGVyYXR1cmUsLi4uZml4ZWQgZWZmZWN0IGluZGljYXRlcyBhIHBhcmFtZXRlciB0aGF0IGlzIGFzc3VtZWQgY29uc3RhbnQsIHdoaWxlIHJhbmRvbSBlZmZlY3RzIGFyZSBwYXJhbWV0ZXJzIHRoYXQgdmFyeSByYW5kb21seSBhcm91bmQgemVybyBhY2NvcmRpbmcgdG8gYSBqb2ludCBtdWx0aXZhcmlhdGUgbm9ybWFsIGRpc3RyaWJ1dGlvbi4KCj4gLi4uSGF2aW5nIGZpeGVkIGVmZmVjdHMgaW4gYW4gKiplY29ub21ldHJpYyBtb2RlbCoqIGhhcyB0aGUgbWVhbmluZyBvZiBhbGxvd2luZyB0aGUgaW50ZXJjZXB0IHRvIHZhcnkgd2l0aCBncm91cCwgb3IgdGltZSwgb3IgYm90aCwgd2hpbGUgdGhlIG90aGVyIHBhcmFtZXRlcnMgYXJlIGdlbmVyYWxseSBzdGlsbCBhc3N1bWVkIHRvIGJlIGhvbW9nZW5lb3VzLiBIYXZpbmcgcmFuZG9tIGVmZmVjdHMgbWVhbnMgaGF2aW5nIGEgZ3JvdXDigJMgKG9yIHRpbWXigJMsIG9yIGJvdGgpIHNwZWNpZmljIGNvbXBvbmVudCBpbiB0aGUgZXJyb3IgdGVybS4KCgpPSywgc28gbm93IHRoYXQgd2Uga25vdyBub3QgdG8gZXhwZWN0IHRoZSB0eXBpY2FsIG1peGVkIG1vZGVsIGRlZmluaXRpb25zIG9mIGZpeGVkIGFuZCByYW5kb20gZWZmZWN0cywgbGV0J3MgZ2V0IGJhY2sgdG8gdGhlIHRocmVlIHN1Yi10eXBlcyBvZiBwYW5lbCByZWdyZXNzaW9uIGFuYWx5c2lzOgoKMSkgKippbmRlcGVuZGVudGx5IHBvb2xlZCBwYW5lbHMqKiAgLSBhc3N1bWVzIHRoYXQgdGhlcmUgYXJlIG5vIGluZGl2aWR1YWwgZWZmZWN0cyB0aGF0IGFyZSBpbmRlcGVuZGVudCBvZiB0aW1lIGFuZCBhbHNvIG5vIGVmZmVjdCBvZiB0aW1lIG9uIGFsbCB0aGUgaW5kaXZpZHVhbHMuIEluIG90aGVyIHdvcmRzLCB0aGUgaW5kZXBlbmRlbnQgdmFyaWFibGVzIGFyZSBub3QgY29ycmVsYXRlZCB3aXRoIGFueSBlcnJvciB0ZXJtLiBUaGlzIGlzIGVzc2VudGlhbGx5IGFuIG9yZGluYXJ5IGxlYXN0IHNxdWFyZXMgbGluZWFyIHJlZ3Jlc3Npb24uIEluIG91ciBzZXR0aW5nLCBpdCB3b3VsZCBtZWFuIHRyZWF0aW5nIGVhY2ggeWVhci1zdGF0ZSBvYnNlcnZhdGlvbiBhcyB1bnJlbGF0ZWQgdG8gdGhlIG90aGVycywgd2hpY2ggcHJvYmFibHkgZG9lcyBub3QgbWFrZSBzZW5zZSwgc2luY2Ugd2UgZXhwZWN0IG9ic2VydmF0aW9ucyB3aXRoaW4gYSBzdGF0ZSB0byBiZSByZWxhdGVkIHRvIG9uZSBhbm90aGVyLiBUaGlzIHR5cGUgb2YgcGFuZWwgcmVncmVzc2lvbiBtYWtlcyB0aGUgbW9zdCBhc3N1bXB0aW9ucyBhbmQgaXMgdGhlcmVmb3JlIHR5cGljYWxseSBub3QgdXNlZCBmb3IgcGFuZWwgZGF0YS4gSW4gdGhpcyBjYXNlIHRoZSBtb2RlbCBmb3JtdWxhdGlvbiBpczoKJCRZX3tpdH09XGJldGFfezB9K1xiZXRhX3sxfXhfezFpdH0rLi4uK1xiZXRhX3tLfVhfe0tpdH0gKyBlX3tpdH0kJAp3aGVyZSB0aGUgaW50ZXJjZXB0ICRcYmV0YV97MGl0fT1cYmV0YV8wJGZvciBhbGwgJGksdCQgYW5kIHNsb3BlICRcYmV0YV97a2l0fT1cYmV0YV9rJCBmb3IgYWxsICRpLHQkLgoKMikgKipmaXhlZCBlZmZlY3RzKiogLSBhc3N1bWVzIHRoYXQgdGhlcmUgYXJlIHVua25vd24gb3IgdW5vYnNlcnZlZCB1bmlxdWUgYXNwZWN0cyBhYm91dCB0aGUgaW5kaXZpZHVhbHMgb3IgaGV0ZXJvZ2VuZWl0eSBhbW9uZyBpbmRpdmlkdWFscyAkYV9pJCB0aGF0IGFyZSBub3QgZXhwbGFpbmVkIGJ5IHRoZSBpbmRlcGVuZGVudCB2YXJpYWJsZXMgYnV0IGluZmx1ZW5jZSB0aGUgb3V0Y29tZSB2YXJpYWJsZSBvZiBpbnRlcmVzdC4gVGhleSBkbyBub3QgdmFyeSB3aXRoIHRpbWUgb3IgaW4gb3RoZXIgd29yZHMgYXJlIGZpeGVkIG92ZXIgdGltZSBidXQgbWF5IGJlICoqY29ycmVsYXRlZCoqIHdpdGggaW5kZXBlbmRlbnQgdmFyaWFibGVzICRYX3tpdH0kLiAgCiAKSW4gdGhpcyBjYXNlIHRoZSBpbnRlcmNlcHQgY2FuIGJlIGRpZmZlcmVudCBmb3IgZWFjaCBpbmRpdmlkdWFsICRcYmV0YV97MGl9JCwgYnV0IHRoZSBvdGhlciBjb2VmZmljaWVudHMgYXJlIGFzc3VtZWQgdG8gYmUgdGhlIHNhbWUgYWNyb3NzIGFsbCB0aGUgaW5kaXZpZHVhbHMuCiAKVGhlc2UgaW5kaXZpZHVhbCAkYV9pJCBlZmZlY3RzIGNhbiBiZSBjb3JyZWxhdGVkIHdpdGggdGhlIGluZGVwZW5kZW50IHZhcmlhYmxlcyAkWCQuIFRoaXMgbW9kZWwgY2FuIGJlIGV4cHJlc3NlZCBhcwokJFlfe2l0fT1cYmV0YV97MH0rXGJldGFfezF9WF97MWl0fSsuLi5cYmV0YV97S31YX3tLaXR9KyBhX2kgK2Vfe2l0fSQkCm9yIGFsdGVybmF0aXZlbHkgdGhlIGluZGl2aWR1YWwgZWZmZWN0cyBjYW4gYmUgYWJzb3JiZWQgaW50byBhbiBpbmRpdmlkdWFsLXNwZWNpZmljIGludGVyY2VwdCB0ZXJtICRcYmV0YV97MGl9JDoKJCRZX3tpdH09XGJldGFfezBpfStcYmV0YV97MX1YX3sxaXR9Ky4uLlxiZXRhX3trfVhfe2tpdH0gK2Vfe2l0fSQkClRoaXMgdHlwZSBvZiBwYW5lbCByZWdyZXNzaW9uIG1ha2VzIHRoZSBmZXdlc3QgYXNzdW1wdGlvbnMuCgoKMykgKipyYW5kb20gZWZmZWN0cyoqIC0gYXNzdW1lcyB0aGF0IHRoZXJlIGFyZSB1bmtub3duIG9yIHVub2JzZXJ2ZWQgdW5pcXVlIHF1YWxpdGllcyBhYm91dCB0aGUgaW5kaXZpZHVhbHMgdGhhdCBpbmZsdWVuY2UgdGhlIG91dGNvbWUgdmFyaWFibGUgb2YgaW50ZXJlc3QgdGhhdCBhcmUgKipub3QgY29ycmVsYXRlZCoqIHdpdGggdGhlIGluZGVwZW5kZW50IHZhcmlhYmxlcy4gVGh1cywgdGhlIHJhbmRvbSBlZmZlY3RzIG1vZGVsIGFjdHVhbGx5IG1ha2VzICoqbW9yZSBhc3N1bXB0aW9ucyoqIHRoYW4gdGhlIGZpeGVkIGVmZmVjdCBtb2RlbC4gV2Ugd2lsbCBub3QgY29uc2lkZXIgdGhpcyBtb2RlbCBoZXJlLgoKVGhlcmUgaXMgcXVpdGUgYSBsb3Qgb2YgZGlzY3Vzc2lvbiBpbiBlY29ub21ldHJpY3MgYXJvdW5kIHdoaWNoIG1vZGVsIGlzIGFwcHJvcHJpYXRlIHRvIHVzZSBhbmQgd2hlbiwgYnV0IHdlIGFyZSBub3QgZ29pbmcgdG8gZHdlbGwgb24gdGhpcyB0b3BpYyBoZXJlLCBhcyBvdXIgbWFpbiBmb2N1cyBpbiB0aGlzIGNhc2Ugc3R1ZHkgaXMgb24gbXVsdGljb2xsaW5lYXJpdHkuIEhvd2V2ZXIsIGhlcmUgYXJlIHNvbWUgYWRkaXRpb25hbCByZWZlcmVuY2VzIG9uIHRoZXNlIHRvcGljcyBmb3IgZnVydGhlciByZWFkaW5nOgpTZWUgW2hlcmVdKGh0dHBzOi8vd3d3LmJhdWVyLnVoLmVkdS9yc3VzbWVsL3BoZC9lYzEtMTUucGRmKSBhbmQgW2hlcmVdKGh0dHBzOi8vc2l0ZXMuZ29vZ2xlLmNvbS9zaXRlL2Vjb25vbWV0cmljc2FjYWRlbXkvZWNvbm9tZXRyaWNzLW1vZGVscy9wYW5lbC1kYXRhLW1vZGVscykgYW5kIFtoZXJlXShodHRwczovL2NyYW4uci1wcm9qZWN0Lm9yZy93ZWIvcGFja2FnZXMvcGxtL3ZpZ25ldHRlcy9wbG1QYWNrYWdlLmh0bWwpIGZvciBtb3JlIGluZm9ybWF0aW9uIGFib3V0IHRoZXNlIGRpZmZlcmVudCBtb2RlbHMuCgoKIyMgKipEb25vaHVlLCBldCBhbC4qKgoqKioKCk9LISBXZSBhcmUgbm93IHJlYWR5IHRvIHN0YXJ0IGFuYWx5emluZyBvdXIgZGF0YSEgCgpJbiBvdXIgY2FzZSwgd2Ugd2lsbCBiZSBwZXJmb3JtaW5nIGEgZml4ZWQgZWZmZWN0IHBhbmVsIHJlZ3Jlc3Npb24gYW5hbHlzaXMsIGFzIHdlIGRvIGluIGZhY3QgdGhpbmsgdGhhdCBzb21lIG9mIHRoZSB1bm9ic2VydmVkIHF1YWxpdGllcyBhYm91dCB0aGUgZGlmZmVyZW50IHN0YXRlcyBtYXkgYmUgY29ycmVsYXRlZCB3aXRoIHNvbWUgb2Ygb3VyIGluZGVwZW5kZW50IHZhcmlhYmxlcy4gRm9yIGV4YW1wbGUsIHRoZSBsZXZlbCBvZiBlY29ub21pYyBvcHBvcnR1bml0eSBtaWdodCBiZSBhbiB1bm9ic2VydmVkIGZlYXR1cmUgYWJvdXQgdGhlIHN0YXRlcyB0aGF0IGluZmx1ZW5jZXMgdmlvbGVudCBjcmltZSByYXRlIGFuZCB3b3VsZCBiZSBwb3NzaWJseSBjb3JyZWxhdGVkIHdpdGggcG92ZXJ0eSByYXRlIGFuZCB1bmVtcGxveW1lbnQuIFRoZXJlIGFyZSBzdGF0aXN0aWNhbCB0ZXN0cyBmb3IgZXZhbHVhdGluZyB3aGljaCBtb2RlbCBpcyB0aGUgbW9zdCBhcHByb3ByaWF0ZSwgd2hpY2ggYXJlIGltcGxlbWVudGVkIGluIHRoZSBgcGxtYCBwYWNrYWdlIHRoYXQgd2Ugd2lsbCBiZSB1c2luZy4KCgpUbyBwZXJmb3JtIG91ciBhbmFseXNpcyB3ZSB3aWxsIGJlIHVzaW5nIHRoZSBgcGxtYCBwYWNrYWdlLiBUaGlzIHN0YW5kcyBmb3IgUGFuZWwgTGluZWFyIE1vZGVsLgoKV2UgbmVlZCB0byB1c2UgYSBzcGVjaWFsIHR5cGUgb2YgZGF0YSB0byB1c2UgdGhpcyBwYWNrYWdlLCBjYWxsZWQgYSBgcGRhdGEuZnJhbWVgIHdoaWNoIGlzIHNob3J0IGZvciBwYW5lbCBkYXRhIGZyYW1lLiBUaGlzIGFsbG93cyB1cyB0byBzcGVjaWZ5IHRoYXQgd2UgYXJlIHVzaW5nIHBhbmVsIGRhdGEgYW5kIHdoYXQgdGhlIHBhbmVsIHN0cnVjdHVyZSBsb29rcyBsaWtlLCBpLmUuLCBob3cgdGhlIGRpZmZlcmVudCBvYnNlcnZhdGlvbnMgYXJlIG1lYW50IHRvIGJlIGdyb3VwZWQgdG9nZXRoZXIsIGJvdGggYnkgc3RhdGUgYW5kIGJ5IHRpbWUuIAoKV2UgbmVlZCB0byBpbmRpY2F0ZSB3aGljaCB2YXJpYWJsZSBzaG91bGQgYmUgdXNlZCB0byBpZGVudGlmeSB0aGUgaW5kaXZpZHVhbHMgaW4gb3VyIHBhbmVsLCBhbmQgd2hhdCB2YXJpYWJsZSBzaG91bGQgYmUgdXNlZCB0byBpZGVudGlmeSB0aGUgdGltZSBwZXJpb2RzIGluIG91ciBwYW5lbC4gSW4gb3VyIGNhc2UgdGhlIGBTVEFURWAgdmFyaWFibGUgaWRlbnRpZmllcyB0aGUgaW5kaXZpZHVhbHMgYW5kIHRoZSBgWUVBUmAgdmFyaWFibGUgaWRlbnRpZmllcyB0aGUgdGltZSBwZXJpb2RzLgoKV2UgY2FuIHNwZWNpZnkgdGhpcyBzdHJ1Y3R1cmUgdXNpbmcgdGhlIGBwZGF0YS5mcmFtZSgpYCBmdW5jdGlvbiBvZiB0aGUgYHBsbWAgcGFja2FnZSwgYnkgdXNpbmcgdGhlIGBpbmRleGAgYXJndW1lbnQsIHdoZXJlIHRoZSBpbmRpdmlkdWFsIHZhcmlhYmxlIGlzIHNwZWNpZmllZCBmaXJzdCBmb2xsb3dlZCBieSB0aGUgdGltZSB2YXJpYWJsZSwgbGlrZSBzbzogYGluZGV4PWMoIkluZGl2aWR1YWxfVmFyaWFibGVfTkFNRSIsICJUaW1lX1BlcmlvZF9WYXJpYWJsZV9OQU1FIilgLgoKYGBge3J9CmRfcGFuZWxfRE9OT0hVRSA8LSBwZGF0YS5mcmFtZShET05PSFVFX0RGLCBpbmRleCA9IGMoIlNUQVRFIiwgIllFQVIiKSkKCmNsYXNzKGRfcGFuZWxfRE9OT0hVRSkKCnNsaWNlX2hlYWQoZF9wYW5lbF9ET05PSFVFLCBuID0gMykKYGBgCgpJbmRlZWQgd2UgaGF2ZSBub3cgY3JlYXRlZCBhIHBkYXRhLmZyYW1lIG9iamVjdCBhbmQgd2UgY2FuIHNlZSB0aGF0IHRoZSByb3cgbmFtZXMgc2hvdyB0aGUgaW5kaXZpZHVhbCBzdGF0ZXMgYW5kIHRpbWUgcGVyaW9kIHllYXJzLgoKT0ssIG5vdyB3ZSBhcmUgcmVhZHkgdG8gcnVuIG91ciBwYW5lbCBsaW5lYXIgbW9kZWwgb24gb3VyIHBhbmVsIGRhdGEgZnJhbWUuIAoKVG8gZG8gc28gd2Ugd2lsbCB1c2UgdGhlIGBwbG0oKWAgZnVuY3Rpb24gYW5kIHdlIHdpbGwgc3BlY2lmeSB0aGUgZm9ybXVsYSBmb3Igb3VyIG1vZGVsLCB3aGVyZSB0aGUgZGVwZW5kZW50IHZhcmlhYmxlIGBWaW9sX2NyaW1lX3JhdGVfMWtfbG9nYCB3aWxsIGJlIG9uIHRoZSBsZWZ0IG9mIG91ciBgfmAgc2lnbiBhbmQgYWxsIG9mIHRoZSBpbmRlcGVuZGVudCB2YXJpYWJsZXMgd2lsbCBiZSBsaXN0ZWQgb24gdGhlIHJpZ2h0IHdpdGggYCtgIHNpZ25zIGluIGJldHdlZW4gZWFjaC4KCkFzIGRpc2N1c3NlZCBhYm92ZSwgdGhlcmUgYXJlIGRpZmZlcmVudCB0eXBlcyBvZiBwYW5lbCBkYXRhIGFuYWx5c2lzLCBhbmQgd2hpY2ggdHlwZSBvZiBtb2RlbCBpcyBmaXQgaXMgY29udHJvbGxlZCBieSB0aGUgYGVmZmVjdGAgYW5kIGBtb2RlbGAgYXJndW1lbnRzIHRvIGBwbG0oKWAuIFNvIHdlIG5lZWQgdG8gc3BlY2lmeSB3aGF0IHR5cGUgb2YgYGVmZmVjdGAgd2Ugd291bGQgbGlrZSB0byBtb2RlbCBhbmQgd2hhdCB0eXBlIG9mIGBtb2RlbGAgd2Ugd291bGQgbGlrZSB0byB1c2UuCgpUaGVyZSBhcmUgdGhyZWUgbWFpbiBvcHRpb25zIGZvciB0aGUgYGVmZmVjdGAgYXJndW1lbnQ6CjEpIGluZGl2aWR1YWwgIC0gbW9kZWwgZm9yIHRoZSBlZmZlY3Qgb2YgaW5kaXZpZHVhbCBpZGVudGl0eQoyKSB0aW1lIC0gbW9kZWwgZm9yIHRoZSBlZmZlY3Qgb2YgdGltZQozKSB0d293YXlzIC0gbWVhbmluZyBtb2RlbGluZyBmb3IgdGhlIGVmZmVjdCBvZiBib3RoIGluZGl2aWR1YWwgaWRlbnRpdHkgYW5kIHRpbWUgIAoKVGhlcmUgYXJlIGZvdXIgbWFpbiBvcHRpb25zIGZvciB0aGUgYG1vZGVsYCBhcmd1bWVudDogIAoxKSBwb29saW5nIC0gc3RhbmRhcmQgcG9vbGVkIG9yZGluYXJ5IGxlYXN0IHNxdWFyZXMgcmVncmVzc2lvbiBtb2RlbCAgCjIpIHdpdGhpbiAtIGZpeGVkIGVmZmVjdHMgbW9kZWwgKHZhcmlhdGlvbiBiZXR3ZWVuIGluZGl2aWR1YWxzIGlzIGlnbm9yZWQsIG1vZGVsIGNvbXBhcmVzIGluZGl2aWR1YWxzIHRvIHRoZW1zZWx2ZXMgYXQgZGlmZmVyZW50IHBlcmlvZHMgb2YgdGltZSkgIAozKSBiZXR3ZWVuIC0gZml4ZWQgZWZmZWN0cyBtb2RlbCAodmFyaWF0aW9uIHdpdGhpbiBpbmRpdmlkdWFscyBmcm9tIG9uZSB0aW1lIHBvaW50IHRvIGFub3RoZXIgaXMgaWdub3JlZCwgbW9kZWwgY29tcGFyZXMgZGlmZmVyZW50IGluZGl2aWR1YWxzIGF0IGVhY2ggcG9pbnQgb2YgdGltZSkgIAo0KSByYW5kb20gLSByYW5kb20gZWZmZWN0cyAoZWFjaCBzdGF0ZSBoYXMgYSBkaWZmZXJlbnQgaW50ZXJjZXB0IGJ1dCBmb3JjZSBpdCB0byBmb2xsb3cgYSBub3JtYWwgZGlzdHJpYnV0aW9uIC0gcmVxdWlyZXMgbW9yZSBhc3N1bXB0aW9ucykKClR5cGljYWxseSBpdCBpcyBiZXN0IHRvIHRoaW5rIGFib3V0IHdoYXQgeW91IGFyZSB0cnlpbmcgdG8gZXZhbHVhdGUgd2l0aCB5b3VyIGRhdGEgaW4gdHJ5aW5nIHRvIGNob29zZSBob3cgdG8gbW9kZWwgeW91ciBkYXRhLiBIb3dldmVyLCB0aGVyZSBhcmUgYWxzbyBzb21lIHRlc3RzIHRoYXQgY2FuIGhlbHAgdG8gYXNzZXNzIHRoaXMgd2hpY2ggd2Ugd2lsbCBicmllZmx5IGNvdmVyLgoKV2UgYXJlIGludGVyZXN0ZWQgaW4gaG93IHZpb2xlbmNlIGluIGVhY2ggc3RhdGUgdmFyaWVkIG92ZXIgdGltZSwgdGh1cyB3ZSBhcmUgaW50ZXJlc3RlZCBpbiB3aXRoaW4gYFNUQVRFYHZhcmlhdGlvbiwgc28gd2Ugd2lsbCBwZXJmb3JtIG91ciBQTE0gYW5hbHlzaXMgd2l0aCB0aGUgYG1vZGVsID0gd2l0aGluYCBhcmd1bWVudCB0byBwZXJmb3JtIHRoaXMgcGFydGljdWxhciB0eXBlIG9mIGZpeGVkIGVmZmVjdHMgbW9kZWwuIAoKV2UgYWxzbyBzcGVjdWxhdGUgdGhhdCB0aGVyZSBpcyBhbiBlZmZlY3Qgb2YgaW5kaXZpZHVhbCBgU1RBVEVgIGlkZW50aXR5IGFuZCB0aW1lIG9uIHZpb2xlbnQgY3JpbWUgcmF0ZS4gSW4gb3RoZXIgd29yZHMsIHdlIGV4cGVjdCBzb21lIHN0YXRlcyB0byBoYXZlIGhpZ2ggcmF0ZXMgb2YgY3JpbWUsIGFuZCBvdGhlcnMgdG8gaGF2ZSBsb3cgcmF0ZXMgb2YgY3JpbWUuIFdlIGFsc28gZXhwZWN0IGNyaW1lIHRvIGNoYW5nZSBvdmVyIHRpbWUuIFRoaXMgbWVhbnMgd2Ugd2FudCB0byB1c2UgdGhlIGBlZmZlY3QgPSAidHdvd2F5cyJgIGFyZ3VtZW50IHRvIGBwbG0oKWAuCgpIZXJlIGlzIGhvdyB3ZSB3b3VsZCBpbXBsZW1lbnQgdGhpcyBmb3IgdGhlIERvbm9odWUgZGF0YToKCmBgYHtyfQpET05PSFVFX09VVFBVVCA8LSBwbG0oVmlvbF9jcmltZV9yYXRlXzFrX2xvZyB+CiAgICAgICAgICAgICAgICAgICAgICBSVENfTEFXICsKICAgICAgICAgICAgICAgICAgICAgIFdoaXRlX01hbGVfMTVfdG9fMTlfeWVhcnMgKwogICAgICAgICAgICAgICAgICAgICAgV2hpdGVfTWFsZV8yMF90b18zOV95ZWFycyArCiAgICAgICAgICAgICAgICAgICAgICBCbGFja19NYWxlXzE1X3RvXzE5X3llYXJzICsKICAgICAgICAgICAgICAgICAgICAgIEJsYWNrX01hbGVfMjBfdG9fMzlfeWVhcnMgKwogICAgICAgICAgICAgICAgICAgICAgT3RoZXJfTWFsZV8xNV90b18xOV95ZWFycyArCiAgICAgICAgICAgICAgICAgICAgICBPdGhlcl9NYWxlXzIwX3RvXzM5X3llYXJzICsKICAgICAgICAgICAgICAgICAgICAgIFVuZW1wbG95bWVudF9yYXRlICsKICAgICAgICAgICAgICAgICAgICAgIFBvdmVydHlfcmF0ZSArCiAgICAgICAgICAgICAgICAgICAgICBQb3B1bGF0aW9uX2xvZyArCiAgICAgICAgICAgICAgICAgICAgICBwb2xpY2VfcGVyXzEwMGtfbGFnLAogICAgICAgICAgICAgICAgICAgICAgZWZmZWN0ID0gInR3b3dheXMiLAogICAgICAgICAgICAgICAgICAgICAgbW9kZWwgPSAid2l0aGluIiwKICAgICAgICAgICAgICAgICAgICAgIGRhdGEgPSBkX3BhbmVsX0RPTk9IVUUKICAgICAgICAgICAgICAgICAgICAgICkKYGBgCgpUbyBzZWUgdGhlIHJlc3VsdHMgd2UgY2FuIHVzZSB0aGUgYmFzZSBgc3VtbWFyeSgpYCBmdW5jdGlvbi4gV2UgY2FuIHZpZXcgdGhpcyBvdXRwdXQgaW4gdGlkeSBmb3JtYXQgdXNpbmcgdGhlIGB0aWR5KClgIGZ1bmN0aW9uIG9mIHRoZSBgYnJvb21gIHBhY2thZ2UuCgpXZSB3aWxsIGFkZCBhbiBgYW5hbHlzaXNgIHZhcmlhYmxlIGFzIGEgbGFiZWwgZm9yIHBsb3RzLgoKYGBge3J9CnN1bW1hcnkoRE9OT0hVRV9PVVRQVVQpCgpET05PSFVFX09VVFBVVF9USURZIDwtIHRpZHkoRE9OT0hVRV9PVVRQVVQsIGNvbmYuaW50ID0gMC45NSkKCkRPTk9IVUVfT1VUUFVUX1RJRFkKCkRPTk9IVUVfT1VUUFVUX1RJRFkkQW5hbHlzaXMgPC0gIkFuYWx5c2lzIDEiCmBgYAoKQXMgZGlzY3Vzc2VkIGFib3ZlLCB3ZSBhcmUgY2hvb3NpbmcgdG8gdXNlIGEgZml4ZWQgZWZmZWN0cyBtb2RlbCBiZWNhdXNlIHdlIGJlbGlldmUgaXQgaXMgbW9zdCBhcHByb3ByaWF0ZSBmb3Igb3VyIGRhdGEuIEhvd2V2ZXIsIHRoZXJlIGFyZSBzdGF0aXN0aWNhbCB0ZXN0cyBpbXBsZW1lbnRlZCBpbiB0aGUgYHBsbWAgcGFja2FnZSB0aGF0IHdvdWxkIGFsbG93IHVzIHRvIHRlc3Qgd2hldGhlciBvbmUgb2YgdGhlIG90aGVyIG1vZGVscyB3b3VsZCBiZSBhcyBhcHByb3ByaWF0ZSBmb3Igb3VyIGRhdGEuIEZvciBleGFtcGxlLCB3ZSBjb3VsZCBwZXJmb3JtIGEgdGVzdCB0byBkZXRlcm1pbmUgaWYgd2UgY291bGQgaGF2ZSBzaW1wbHkgdXNlZCBhIHBvb2xlZCBtb2RlbCwgaS5lLiwgdGhlIHRlc3QgZXZhbHVhdGVzIHdoZXRoZXIgdGhlIGNvZWZmaWNpZW50cyAoaW5jbHVkaW5nIHRoZSBpbnRlcmNlcHRzKSBhcmUgZXF1YWwgYWNyb3NzIGluZGl2aWR1YWxzLiBUaGlzIHRlc3QgaXMgaW1wbGVtZW50ZWQgaW4gdGhlIGBwb29sdGVzdCgpYCBmdW5jdGlvbiBvZiB0aGUgYHBsbWAgcGFja2FnZSBhbmQgcGVyZm9ybXMgYSBjb21wYXJpc29uIGJldHdlZW4gdGhlIHBvb2xlZCBtb2RlbCB0byB0aGUgZml4ZWQgZWZmZWN0IHdpdGhpbiBtb2RlbC4gVG8gdGVzdCBpZiB1c2luZyBhIHJhbmRvbSBlZmZlY3QgbW9kZWwgd291bGQgYmUgbW9yZSBhcHByb3ByaWF0ZSBjb21wYXJlZCB0byB0aGUgZml4ZWQgZWZmZWN0IG1vZGVsLCBvbmUgY291bGQgdXNlIHRoZSBbSGF1c21hbiB0ZXN0XShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9EdXJiaW4lRTIlODAlOTNXdSVFMiU4MCU5M0hhdXNtYW5fdGVzdCl7dGFyZ2V0PSJfYmxhbmsifSAoYWxzbyBjYWxsZWQgdGhlIER1cmJpbi1XdS1IYXVzbWFuIHRlc3QpLiBUaGlzIHRlc3QgaXMgaW1wbGVtZW50ZWQgdXNpbmcgdGhlIGBwaHRlc3QoKWAgZnVuY3Rpb24gb2YgdGhlIGBwbG1gIHBhY2thZ2UuCgpGaW5hbGx5LCB3ZSBjb3VsZCBhbHNvIHBlcmZvcm0gYSB0ZXN0IHRvIGV2YWx1YXRlIGlmIHRoZXJlIGlzIGluZGVlZCBhbiBpbmRpdmlkdWFsIChzdGF0ZSkgZWZmZWN0IGFuZCBhIHRpbWUgZWZmZWN0IGluIG91ciBtb2RlbCB1c2luZyB0aGUgYHBsbXRlc3QoKWAgZnVuY3Rpb24gb2YgdGhlIGBwbG1gIHBhY2thZ2UuIAoKRm9yIG1vcmUgaW5mb3JtYXRpb24gb24gdGhlc2UgdGVzdHMgYW5kIHRoaXMgcGFja2FnZSwgc2VlIFtoZXJlXShodHRwczovL2NyYW4uci1wcm9qZWN0Lm9yZy93ZWIvcGFja2FnZXMvcGxtL3ZpZ25ldHRlcy9wbG1QYWNrYWdlLmh0bWwpe3RhcmdldD0iX2JsYW5rIn0gIGFuZCBbaGVyZV0oaHR0cDovL3d3dy5wcmluY2V0b24uZWR1L35vdG9ycmVzL1BhbmVsMTAxUi5wZGYpe3RhcmdldD0iX2JsYW5rIn0uCgoKIyMgKipNdXN0YXJkIGFuZCBMb3R0KioKKioqCgpPSywgbm93IHdlIHdpbGwgZG8gdGhlIHNhbWUgZm9yIHRoZSBNdXN0YXJkIGFuZCBMb3R0IGFuYWx5c2lzLiBJbiB0aGlzIGNhc2Ugd2Ugd291bGQgaGF2ZSBhIHZlcnkgbGFyZ2UgZm9ybXVsYSB0byB3cml0ZS4gU28gaW5zdGVhZCwgd2UgY2FuIHVzZSB0aGUgYGFzLmZvcm11bGEoKWAgZnVuY3Rpb24gb2YgdGhlIGBzdGF0c2AgcGFja2FnZSBhbmQgdGhlIGJhc2UgYHBhc3RlKClgIGZ1bmN0aW9uIHRvIGNvbWJpbmUgYWxsIG9mIG91ciBleHBsYW5hdG9yeSB2YXJpYWJsZXMgaW50byBvbmUgZm9ybXVsYSB3aXRob3V0IG1ha2luZyBhIG1pc3Rha2UuIEZpcnN0IHdlIHdpbGwgY3JlYXRlIGFuIG9iamVjdCB3aGVyZSB3ZSBzZWxlY3Qgb25seSB0aGUgZXhwbGFuYXRvcnkgdmFyaWFibGVzLiAKCmBgYHtyfQpMT1RUX3ZhcmlhYmxlcyA8LSBMT1RUX0RGICU+JQogIGRwbHlyOjpzZWxlY3QoCiAgICBSVENfTEFXLAogICAgY29udGFpbnMoYygiV2hpdGUiLCAiQmxhY2siLCAiT3RoZXIiKSksCiAgICBVbmVtcGxveW1lbnRfcmF0ZSwKICAgIFBvdmVydHlfcmF0ZSwKICAgIFBvcHVsYXRpb25fbG9nLAogICAgcG9saWNlX3Blcl8xMDBrX2xhZwogICkgJT4lCiAgY29sbmFtZXMoKQoKCkxPVFRfZm1sYSA8LSBhcy5mb3JtdWxhKHBhc3RlKAogICJWaW9sX2NyaW1lX3JhdGVfMWtfbG9nIH4iLAogIHBhc3RlKExPVFRfdmFyaWFibGVzLCBjb2xsYXBzZSA9ICIgKyAiKQopKQoKTE9UVF9mbWxhCmBgYAoKVGhhdCBpcyBxdWl0ZSB0aGUgZm9ybXVsYSEKCk9LLCBub3cgYWdhaW4gd2Ugd2lsbCBtYWtlIGEgcGFuZWwgZGF0YSBmcmFtZSBhbmQgd2Ugd2lsbCBmaXQgYSBmaXhlZCBlZmZlY3QgdHdvLXdheSBtb2RlbCBmb3IgdGltZSBhbmQgaW5kaXZpZHVhbHMgKGBTVEFURWApIHdpdGggdGhpcyBkYXRhIGFzIHdlbGwuCgpgYGB7cn0KCmRfcGFuZWxfTE9UVCA8LSBwZGF0YS5mcmFtZShMT1RUX0RGLCBpbmRleCA9IGMoIlNUQVRFIiwgIllFQVIiKSkKCkxPVFRfT1VUUFVUIDwtIHBsbShMT1RUX2ZtbGEsCiAgbW9kZWwgPSAid2l0aGluIiwKICBlZmZlY3QgPSAidHdvd2F5cyIsCiAgZGF0YSA9IGRfcGFuZWxfTE9UVAopCgpzdW1tYXJ5KExPVFRfT1VUUFVUKQoKTE9UVF9PVVRQVVRfVElEWSA8LSB0aWR5KExPVFRfT1VUUFVULCBjb25mLmludCA9IDAuOTUpCgpMT1RUX09VVFBVVF9USURZJEFuYWx5c2lzIDwtICJBbmFseXNpcyAyIgpgYGAKCgojIyAqKlJUQyBjb2VmZmljaWVudCBjb21wYXJpc29uKioKKioqCgpOb3cgbGV0J3MgbWFrZSBhIHBsb3QgdG8gY29tcGFyZSB0aGUgY29lZmZpY2llbnQgZXN0aW1hdGUgZm9yIHRoZSBSaWdodC10by1jYXJyeSBsYXcgYWRvcHRpb24gdmFyaWFibGUgaW4gZWFjaCBtb2RlbC4KCkZpcnN0IHdlIHdpbGwgY29tYmluZSBtb2RlbCBmaXQgaW5mb3JtYXRpb24gZm9yIHRoaXMgY29lZmZpY2llbnQgZm9yIGVhY2ggbW9kZWwuCgpgYGB7cn0KY29tcGFyaW5nX2FuYWx5c2VzIDwtIERPTk9IVUVfT1VUUFVUX1RJRFkgJT4lCiAgYmluZF9yb3dzKExPVFRfT1VUUFVUX1RJRFkpICU+JQogIGZpbHRlcih0ZXJtID09ICJSVENfTEFXVFJVRSIpCgpjb21wYXJpbmdfYW5hbHlzZXMKYGBgCgpXZSBjYW4gc2VlIHRoYXQgZm9yIHRoZSBmaXJzdCBhbmFseXNpcyAoc2ltaWxhciB0byB0aGUgW0Rvbm9odWUgZXQgYWwuXShodHRwczovL3d3dy5uYmVyLm9yZy9wYXBlcnMvdzIzNTEwLnBkZil7dGFyZ2V0PSJfYmxhbmsifSBzdHVkeSkgdGhlIGNvZWZmaWNpZW50IGVzdGltYXRlIGZvciB0aGUgcHJlc2VuY2Ugb2YgYSBwZXJtaXNzaXZlIHJpZ2h0LXRvLWNhcnJ5IGxhdyBpcyBwb3NpdGl2ZSwgd2hpbGUgZm9yIHRoZSBzZWNvbmQgYW5hbHlzaXMgKHNpbWlsYXIgdG8gdGhlIFtNdXN0YXJkIGFuZCBMb3R0XShodHRwczovL2NoaWNhZ291bmJvdW5kLnVjaGljYWdvLmVkdS9jZ2kvdmlld2NvbnRlbnQuY2dpP2FydGljbGU9MTE1MCZjb250ZXh0PWxhd19hbmRfZWNvbm9taWNzKXt0YXJnZXQ9Il9ibGFuayJ9IHN0dWR5KSB0aGUgY29lZmZpY2llbnQgZXN0aW1hdGUgaXMgbmVnYXRpdmUuIFRodXMgaW4gdGhlIGZpcnN0IGFuYWx5c2lzIHdlIGNvdWxkIGNvbmNsdWRlIHRoYXQgdGhlIGVmZmVjdCBvZiBhZG9wdGluZyBwZXJtaXNzaXZlIHJpZ2h0LXRvLWNhcnJ5IGxhd3MgIG1heSBiZSBhc3NvY2lhdGVkIHdpdGggaW5jcmVhc2VzIGluIHZpb2xlbnQgY3JpbWUgKGFsdGhvdWdoIHRoaXMgd2FzIG5vdCBhIHNpZ25pZmljYW50IHJlc3VsdCAoaW4gY29udHJhc3Qgd2l0aCB0aGUgcmVhbCBbRG9ub2h1ZSBldCBhbC5dKGh0dHBzOi8vd3d3Lm5iZXIub3JnL3BhcGVycy93MjM1MTAucGRmKXt0YXJnZXQ9Il9ibGFuayJ9IHN0dWR5ICkpOyB3aGlsZSBpbiB0aGUgb3RoZXIgYW5hbHlzaXMgd2UgY291bGQgY29uY2x1ZGUgdGhhdCB0aGUgbGF3cyBtYXkgYmUgYXNzb2NpYXRlZCB3aXRoIGRlY3JlYXNlcyBpbiB2aW9sZW50IGNyaW1lLgoKTGV0J3MgbWFrZSBhIHBsb3Qgb2YgdGhpcyBmaW5kaW5nLiBXZSB3aWxsIHNob3cgZXJyb3IgYmFycyBmb3IgdGhlIGNvZWZmaWNpZW50IGVzdGltYXRlcyBmb3IgYm90aCBhbmFseXNlcyB1c2luZyB0aGUgYGdlb21fZXJyb3JiYXIoKWAgZnVuY3Rpb24gb2YgdGhlIGBnZ3Bsb3QyYCBwYWNrYWdlLiBUaGlzIHJlcXVpcmVzIHNwZWNpZnlpbmcgdGhlIG1pbmltdW0gYW5kIG1heGltdW0gZm9yIG91ciBlcnJvciBiYXIsIHdoaWNoIGluIG91ciBjYXNlIHdpbGwgYmUgdGhlIGxvdyBhbmQgaGlnaCB2YWx1ZXMgb2Ygb3VyIGNvbmZpZGVuY2UgaW50ZXJ2YWxzIGZvciB0aGUgY29lZmZpY2llbnQgZXN0aW1hdGVzLiBXZSB3aWxsIGFsc28gYWRkIGEgaG9yaXpvbnRhbCBsaW5lIGF0IHkgPSAwIHVzaW5nIHRoZSBgZ2VvbV9obGluZSgpYCBmdW5jdGlvbiBvZiB0aGUgYGdncGxvdDJgIHBhY2thZ2UuIAoKRmluYWxseSB3ZSB3aWxsIGFkZCBhcnJvd3MgdG8gZW1waGFzaXplIHRoZSBkaWZmZXJlbmNlIGluIHRoZSBkaXJlY3Rpb24gb2YgdGhlIGZpbmRpbmdzIHVzaW5nIHRoZSBgZ2VvbV9zZWdtZW50KClgIGZ1bmN0aW9uIG9mIHRoZSBgZ2dwbG90MmAgcGFja2FnZS4gVXNpbmcgdGhlIGBhcnJvdygpYCBmdW5jdGlvbiwgd2UgY2FuIHNwZWNpZnkgZGV0YWlscyBhYm91dCB0aGUgYXJyb3cgd2Ugd291bGQgbGlrZSB0byBhZGQuCgpgYGB7cn0KY29tcGFyaW5nX2FuYWx5c2VzX3Bsb3QgPC0gZ2dwbG90KGNvbXBhcmluZ19hbmFseXNlcykgKwogIGdlb21fcG9pbnQoYWVzKHggPSBBbmFseXNpcywgeSA9IGVzdGltYXRlKSkgKwogIGdlb21fZXJyb3JiYXIoYWVzKHggPSBBbmFseXNpcywgeW1pbiA9IGNvbmYubG93LCB5bWF4ID0gY29uZi5oaWdoKSwgd2lkdGggPSAwLjI1KSArCiAgZ2VvbV9obGluZSh5aW50ZXJjZXB0ID0gMCwgY29sb3IgPSAicmVkIikgKwogIHNjYWxlX3lfY29udGludW91cygKICAgIGJyZWFrcyA9IHNlcSgtMC4yLCAwLjIsIGJ5ID0gMC4wNSksCiAgICBsYWJlbHMgPSBzZXEoLTAuMiwgMC4yLCBieSA9IDAuMDUpLAogICAgbGltaXRzID0gYygtMC4yLCAwLjIpCiAgKSArCiAgZ2VvbV9zZWdtZW50KGFlcyh4ID0gMSwgeSA9IDAuMTI1LCB4ZW5kID0gMSwgeWVuZCA9IDAuMTc1KSwKICAgIGFycm93ID0gYXJyb3coYW5nbGUgPSA0NSwgZW5kcyA9ICJsYXN0IiwgdHlwZSA9ICJvcGVuIiksCiAgICBzaXplID0gMiwKICAgIGNvbG9yID0gImdyZWVuIiwKICAgIGxpbmVlbmQgPSAiYnV0dCIsCiAgICBsaW5lam9pbiA9ICJtaXRyZSIKICApICsKICBnZW9tX3NlZ21lbnQoYWVzKHggPSAyLCB5ID0gLTAuMTI1LCB4ZW5kID0gMiwgeWVuZCA9IC0wLjE3NSksCiAgICBhcnJvdyA9IGFycm93KGFuZ2xlID0gNDUsIGVuZHMgPSAibGFzdCIsIHR5cGUgPSAib3BlbiIpLAogICAgc2l6ZSA9IDIsCiAgICBjb2xvciA9ICJyZWQiLAogICAgbGluZWVuZCA9ICJidXR0IiwKICAgIGxpbmVqb2luID0gIm1pdHJlIgogICkgKwogIHRoZW1lX21pbmltYWwoKSArCiAgdGhlbWUoCiAgICBheGlzLnRpdGxlLnggPSBlbGVtZW50X2JsYW5rKCksCiAgICBheGlzLnRleHQgPSBlbGVtZW50X3RleHQoc2l6ZSA9IDgsIGNvbG9yID0gImJsYWNrIikKICApICsKICBsYWJzKAogICAgdGl0bGUgPSAiRWZmZWN0IGVzdGltYXRlIG9uIGxuKHZpb2xlbnQgY3JpbWVzIHBlciAxMDAsMDAwIHBlb3BsZSkiLAogICAgeSA9ICIgIEVmZmVjdCBlc3RpbWF0ZSAoOTUlIENJKSIKICApCgpjb21wYXJpbmdfYW5hbHlzZXNfcGxvdApgYGAKCldlIGNhbiBzZWUgdGhhdCB0aGUgY29uZmlkZW5jZSBpbnRlcnZhbCBmcm9tIGFuYWx5c2lzIDEgaXMgbW9zdGx5IGNvdmVyaW5nIHBvc2l0aXZlIHZhbHVlcywgd2hpbGUgdGhlIGVudGlyZSBjb25maWRlbmNlIGludGVydmFsIGlzIG5lZ2F0aXZlIGZvciBhbmFseXNpcyAyLgoKIyAqKk11bHRpY29sbGluZWFyaXR5IGFuYWx5c2lzKioKKioqCgpIb3cgZGlkIHRoZSBhYm92ZSBoYXBwZW4/CgpUaGUgYW5hbHlzaXMgZGF0YSBmcmFtZXMgYXJlIHZlcnkgc2ltaWxhciB5ZXQgeWllbGRlZCB2ZXJ5IGRpZmZlcmVudCByZXN1bHRzLiAKClJlY2FsbCB0aGF0IHRoZSBvbmx5IGRpZmZlcmVuY2UgYmV0d2VlbiB0aGUgdHdvIG1vZGVscyBpcyB0aGUgbnVtYmVyIG9mIGRlbW9ncmFwaGljIHZhcmlhYmxlcyBpbmNsdWRlZCBhcyBjb3ZhcmlhdGVzLiBUaGUgbnVtYmVyIG9mIHJvd3Mgb3Igb2JzZXJ2YXRpb25zIGlzIHRoZSBzYW1lLCBhcyBhcmUgdGhlIG91dGNvbWUgYW5kIHRoZSBvdGhlciBjb3ZhcmlhdGVzIGluY2x1ZGVkIGluIHRoZSBtb2RlbC4gV2UgY2FuIHVzZSB0aGUgYGFsbF9lcXVhbCgpYCBmdW5jdGlvbiBvZiB0aGUgYGRwbHlyYCBwYWNrYWdlIHRvIGNvbXBhcmUgdGhlIG51bWJlciBvZiBjb2x1bW5zIG9mIG91ciBEb25vaHVlLWxpa2UgZGF0YSBhbmQgb3VyIE11c3RhcmQgYW5kIExvdHQtbGlrZSBkYXRhLgoKYGBge3J9CmFsbF9lcXVhbCgKICB0YXJnZXQgPSBET05PSFVFX0RGLAogIGN1cnJlbnQgPSBMT1RUX0RGLAogIGlnbm9yZV9jb2xfb3JkZXIgPSBUUlVFLAogIGlnbm9yZV9yb3dfb3JkZXIgPSBUUlVFCikKYGBgCgpVc2luZyB0aGUgYmFzZSBgZGltKClgIGZ1bmN0aW9uIHdlIGNhbiBhbHNvIGxvb2sgYXQgdGhlIG51bWJlciBvZiByb3dzIGZvciBlYWNoIGFuZCBzZWUgdGhhdCB0aGUgbnVtYmVyIG9mIG9ic2VydmF0aW9ucyBpcyB0aGUgc2FtZSBmb3IgYm90aCBkYXRhc2V0cy4KCmBgYHtyfQpkaW0oRE9OT0hVRV9ERilbMV0KZGltKExPVFRfREYpWzFdCmBgYAoKVGhlIG9ubHkgZGlmZmVyZW5jZSBiZXR3ZWVuIHRoZSB0d28gZGF0YSBmcmFtZXMgcmVzdHMgaW4gaG93IHRoZSBkZW1vZ3JhcGhpYyB2YXJpYWJsZXMgd2VyZSBwYXJhbWV0ZXJpemVkLgoKYGBge3J9CkRPTk9IVUVfREYgJT4lCiAgZHBseXI6OnNlbGVjdChjb250YWlucygieWVhcnMiKSkgJT4lCiAgY29sbmFtZXMoKQoKTE9UVF9ERiAlPiUKICBkcGx5cjo6c2VsZWN0KGNvbnRhaW5zKCJ5ZWFycyIpKSAlPiUKICBjb2xuYW1lcygpCmBgYAoKQ2xlYXJseSwgdGhpcyBoYWQgYW4gZWZmZWN0IG9uIHRoZSByZXN1bHRzIG9mIHRoZSBhbmFseXNpcy4gCgpMZXQncyBleHBsb3JlIGhvdyB0aGlzIG9jY3VycmVkLiAKCldoZW4gc2VlbWluZ2x5IGluZGVwZW5kZW50IHZhcmlhYmxlcyBhcmUgaGlnaGx5IHJlbGF0ZWQgdG8gb25lIGFub3RoZXIsIHRoZSByZWxhdGlvbnNoaXBzIGVzdGltYXRlZCBpbiBhbiBhbmFseXNpcyBtYXkgYmUgZGlzdG9ydGVkLiAKCkluIHJlZ3Jlc3Npb24gYW5hbHlzaXMsIHRoaXMgZGlzdG9ydGlvbiBpcyBvZnRlbiBhIGJ5LXByb2R1Y3Qgb2YgYSB2aW9sYXRpb24gb2YgdGhlIGluZGVwZW5kZW5jZSBhc3N1bXB0aW9uLiBUaGlzIGRpc3RvcnRpb24sIGlmIGxhcmdlIGVub3VnaCwgY2FuIGltcGFjdCBzdGF0aXN0aWNhbCBpbmZlcmVuY2UuIAoKVGhlIHBoZW5vbWVub24gY2FsbGVkIG11bHRpY29sbGluZWFyaXR5IG9jY3VycyB3aGVuIGluZGVwZW5kZW50IHZhcmlhYmxlcyBhcmUgaGlnaGx5IHJlbGF0ZWQgdG8gb25lIGFub3RoZXIuCgpUaGVyZSBhcmUgc2V2ZXJhbCB3YXlzIHdlIGNhbiBkaWFnbm9zZSBtdWx0aWNvbGxpbmVhcml0eS4KCiMjIyAqKkNvcnJlbGF0aW9uKioKKioqCgpPbmUgd2F5IHdlIGNhbiBldmFsdWF0ZSB0aGUgcmVsYXRpb25zaGlwcyBiZXR3ZWVuIHZhcmlhYmxlcyBpcyBieSBleGFtaW5pbmcgdGhlIGNvcnJlbGF0aW9uIGJldHdlZW4gdmFyaWFibGUgcGFpcnMuCgo8c3R5bGU+CmRpdi5ibHVlIHsgYmFja2dyb3VuZC1jb2xvcjojZTZmMGZmOyBib3JkZXItcmFkaXVzOiA1cHg7IHBhZGRpbmc6IDIwcHg7fQo8L3N0eWxlPgo8ZGl2IGNsYXNzID0gImJsdWUiPgoKSXQgaXMgaW1wb3J0YW50IHRvIG5vdGUgdGhhdCBtdWx0aWNvbGxpbmVhcml0eSBhbmQgY29ycmVsYXRpb24gYXJlIG5vdCB0aGUgc2FtZSBjb25jZXB0LiBbQ29ycmVsYXRpb25dKGh0dHBzOi8vd3d3Lm5jYmkubmxtLm5paC5nb3YvcG1jL2FydGljbGVzL1BNQzUwNzkwOTMvKSBjYW4gYmUgdGhvdWdodCBvZiBhcyB0aGUgc3RyZW5ndGggb2YgYSBsaW5lYXIgcmVsYXRpb25zaGlwIGJldHdlZW4gdmFyaWFibGVzIGluIGdlbmVyYWwuIE9uIHRoZSBvdGhlciBoYW5kLCB3ZSB1c3VhbGx5IHVzZSB0aGUgdGVybSBjb2xsaW5lYXJpdHkgaW4gdGhlIGNvbnRleHQgb2YgYSBtdWx0aXBsZSByZWdyZXNzaW9uIG1vZGVsLCB3aGVyZSB0d28gaW5kZXBlbmRlbnQgdmFyaWFibGVzIGFyZSBjb2xsaW5lYXIgaWYgdGhleSBoYXZlIGEgbGluZWFyIHJlbGF0aW9uc2hpcCBvciBhc3NvY2lhdGlvbi4gTXVsdGljb2xsaW5lYXJpdHkgY2FuIGJlIHRob3VnaHQgb2YgYXMgY29sbGluZWFyaXR5IGFtb25nIG11bHRpcGxlICgzKykgcmVncmVzc29ycyAoaW5kZXBlbmRlbnQgdmFyaWFibGVzKSBpbiBhIHJlZ3Jlc3Npb24gYW5hbHlzaXMsIHdoaWNoIGNhbiBvY2N1ciB3aGVuIHJlZ3Jlc3NvcnMgYXJlIGhpZ2hseSBjb3JyZWxhdGVkLgoKPC9kaXY+CgoKQWNjb3JkaW5nIHRvIFtXaWtpcGVkaWFdKGh0dHBzOi8vZW4ud2lraXBlZGlhLm9yZy93aWtpL011bHRpY29sbGluZWFyaXR5KXt0YXJnZXQ9Il9ibGFuayJ9OgoKPiBtdWx0aWNvbGxpbmVhcml0eSAoYWxzbyBjb2xsaW5lYXJpdHkpIGlzIGEgcGhlbm9tZW5vbiBpbiB3aGljaCBvbmUgcHJlZGljdG9yIHZhcmlhYmxlIGluIGEgbXVsdGlwbGUgcmVncmVzc2lvbiBtb2RlbCBjYW4gYmUgbGluZWFybHkgcHJlZGljdGVkIGZyb20gdGhlIG90aGVycyB3aXRoIGEgc3Vic3RhbnRpYWwgZGVncmVlIG9mIGFjY3VyYWN5LiAKClRodXMgY29sbGluZWFyaXR5IGRlc2NyaWJlcyBsaW5lYXIgcHJlZGljdGlvbiBvciBhc3NvY2lhdGlvbiBiZXR3ZWVuIHZhcmlhYmxlcy4gT2Z0ZW4gdGhvc2UgdmFyaWFibGVzIHdpbGwgYmUgaGlnaGx5IGNvcnJlbGF0ZWQuCgpUaGUgaXNzdWUgd2l0aCB0aGlzIGluIGxpbmVhciByZWdyZXNzaW9uLCBpcyB0aGF0IGxpbmVhciByZWdyZXNzaW9uIGFpbXMgdG8gZGV0ZXJtaW5lIGhvdyBhIG9uZSB1bml0IGNoYW5nZSBpbiBhIHJlZ3Jlc3NvciBpbmZsdWVuY2VzIGEgb25lIHVuaXQgY2hhbmdlIGluIHRoZSBkZXBlbmRlbnQgdmFyaWFibGUuIEluIGZhY3QsIHRoaXMgaXMgd2hhdCB0aGUgY29lZmZpY2llbnQgZXN0aW1hdGVzIGFpbSB0byB0ZWxsIHVzIGZvciBlYWNoIHJlZ3Jlc3Nvci4KCkhvd2V2ZXIsIGlmIG91ciByZWdyZXNzb3JzIGFyZSBhbHNvIGxpbmVhcmx5IHJlbGF0ZWQgb3IgY29ycmVsYXRlZCwgdGhlbiBpdCBiZWNvbWVzIGRpZmZpY3VsdCB0byBkZXRlcm1pbmUgdGhlIGVmZmVjdCBvZiBlYWNoIHJlZ3Jlc3NvciBvbiB0aGUgZGVwZW5kZW50IHZhcmlhYmxlLCB2YXJpYWJsZSBpbmRlcGVuZGVudCBvZiB0aGUgb3RoZXIgcmVncmVzc29ycywgYW5kIG11bHRpY29sbGluZWFyaXR5IGNhbiBjYXVzZSBpbnN0YWJpbGl0eSBpbiBvdXIgW2NvZWZmaWNpZW50IGVzdGltYXRlc10oaHR0cHM6Ly9zdGF0aXN0aWNzYnlqaW0uY29tL2dsb3NzYXJ5L3JlZ3Jlc3Npb24tY29lZmZpY2llbnQvKXt0YXJnZXQ9Il9ibGFuayJ9LCBtYWtpbmcgdGhlbSB1bnJlbGlhYmxlLiBJbiB0aGUgcHJlc2VuY2Ugb2YgbXVsdGljb2xsaW5lYXJpdHksIGNvZWZmaWNpZW50IGVzdGltYXRlcyBtYXkgYmUgaW5mbGF0ZWQsIGRlZmxhdGVkLCBvciB0aGVpciBzaWducyBtYXkgY2hhbmdlLiAKCkZvciBleGFtcGxlLCBzYXkgYm90aCB3YWlzdCBhbmQgaGlwIGNpcmN1bWZlcmVuY2UgYXJlIGluY2x1ZGVkIGFzIHByZWRpY3RvcnMgb2YgQk1JIGluIGEgbXVsdGlwbGUgcmVncmVzc2lvbiBtb2RlbC4gQXMgaW4gYW55IG11bHRpcGxlIHJlZ3Jlc3Npb24gbW9kZWwsIHRoZSBjb2VmZmljaWVudCBvZiB3YWlzdCBjaXJjdW1mZXJlbmNlIHJlcHJlc2VudHMgdGhlIGF2ZXJhZ2UgY2hhbmdlIGluIEJNSSBmb3IgYSBvbmUgdW5pdCBjaGFuZ2UgaW4gd2Fpc3QgY2lyY3VtZmVyZW5jZSwgaG9sZGluZyB0aGUgb3RoZXIgcHJlZGljdG9ycyBpbiB0aGUgbW9kZWwgY29uc3RhbnQuIFNpbmNlIGl0IGlzIG5vdCB2ZXJ5IG1lYW5pbmdmdWwgdG8gY29uc2lkZXIgdmFyaWF0aW9uIGluIHdhaXN0IGNpcmN1bWZlcmVuY2Ugd2hpbGUgaG9sZGluZyBoaXAgY2lyY3VtZmVyZW5jZSBjb25zdGFudCAoYXMgd2Fpc3QgYW5kIGhpcCBjaXJjdW1mZXJlbmNlIGFyZSBpbmRlZWQgcmVsYXRlZCB0byBvbmUgYW5vdGhlciksIG91ciBlc3RpbWF0ZSBvZiB0aGlzIGNvZWZmaWNpZW50IG1heSBiZSB1bmNlcnRhaW4uIAoKCklmIHlvdSB3YW50IHRvIHJlYWQgZnVydGhlciBvbiB0aGlzIHRvcGljLCBbaGVyZV0oaHR0cHM6Ly9ibG9nLmNsYWlydm95YW50c29mdC5jb20vY29ycmVsYXRpb24tYW5kLWNvbGxpbmVhcml0eS1ob3ctdGhleS1jYW4tbWFrZS1vci1icmVhay1hLW1vZGVsLTkxMzVmYmU2OTM2YSl7dGFyZ2V0PSJfYmxhbmsifSBhbmQgW2hlcmVdKGh0dHBzOi8vbWVkaXVtLmNvbS9zd2xoL211bHRpY29sbGluZWFyaXR5LWFuZC12YXJpYW5jZS1pbmZsYXRpb24tZmFjdG9yLWJjNzRhZjM2YjFjOSl7dGFyZ2V0PSJfYmxhbmsifSBhcmUgYSBjb3VwbGUgb2YgaW50ZXJlc3RpbmcgZGlzY3Vzc2lvbnMuCgpJbiB0aGUgbmV4dCBzZWN0aW9ucywgd2Ugd2lsbCBkZXNjcmliZSB3YXlzIHRvIGRldGVjdCBtdWx0aWNvbGxpbmVhcml0eSBpbiBvdXIgY292YXJpYXRlcyBib3RoIHVzaW5nIHZpc3VhbCBkaXNwbGF5cyBvZiBkYXRhIGFuZCB1c2luZyBjb21wdXRhdGlvbmFsIHRlY2huaXF1ZXMuCgojIyMjIFNjYXR0ZXIgcGxvdHMKCk9uZSBvZiB0aGUgd2F5cyB0byBkaWFnbm9zZSBtdWx0aWNvbGxpbmVhcml0eSBpbiBhIHJlZ3Jlc3Npb24gbW9kZWwgaXMgdG8gZmlyc3Qgc2VlIGlmIHRoZXJlIGFyZSByZWdyZXNzb3JzIHRoYXQgYXJlIGhpZ2hseSBjb3JyZWxhdGVkLiBJZiBzbywgdGhpcyBzdWdnZXN0cyB0aGF0IHdlIHNob3VsZCBpbnZlc3RpZ2F0ZSBmdXJ0aGVyIHRvIHNlZSBpZiB0aGVzZSB2YXJpYWJsZXMgYXJlIGluIGZhY3QgbGluZWFybHkgcHJlZGljdGluZyBvbmUgYW5vdGhlci4KCk9uZSB3YXkgdG8gbG9vayBhdCBjb3JyZWxhdGlvbiBhY3Jvc3MgcGFpcnMgb2YgdmFyaWFibGVzIGlzIHRvIHVzZSB0aGUgYGdncGFpcnMoKWAgZnVuY3Rpb24gb2YgdGhlIGBHR2FsbHlgIHBhY2thZ2UuCgpgYGB7cn0KY29sbmFtZXMoRE9OT0hVRV9ERikKCkRPTk9IVUVfREYgJT4lCiAgZHBseXI6OnNlbGVjdCgKICAgIFJUQ19MQVcsCiAgICBWaW9sX2NyaW1lX3JhdGVfMWtfbG9nLAogICAgVW5lbXBsb3ltZW50X3JhdGUsCiAgICBQb3ZlcnR5X3JhdGUsCiAgICBQb3B1bGF0aW9uX2xvZwogICkgJT4lCiAgZ2dwYWlycyguLAogICAgY29sdW1ucyA9IGMoMjo1KSwKICAgIGxvd2VyID0gbGlzdChjb250aW51b3VzID0gd3JhcCgic21vb3RoX2xvZXNzIiwKICAgICAgY29sb3IgPSAicmVkIiwKICAgICAgYWxwaGEgPSAwLjUsCiAgICAgIHNpemUgPSAwLjEKICAgICkpCiAgKQpgYGAKV2UgY2FuIHNlZSB0aGF0IGZvciB0aGUgbm9uLWRlbW9ncmFwaGljIHZhcmlhYmxlcywgdGhlcmUgaXMgdmVyeSBsaXR0bGUgY29ycmVsYXRpb24gYmV0d2VlbiB0aGUgcGFpcnMgb2YgdmFyaWFibGVzLiBPbmx5IHRoZSB1bmVtcGxveW1lbnQgcmF0ZSBhbmQgdGhlIHBvdmVydHkgcmF0ZSBzaG93IHJlbGF0aXZlbHkgc3Ryb25nIGNvcnJlbGF0aW9uLCBhcyBvbmUgbWlnaHQgZXhwZWN0LiAKCgojIyMjIEhlYXRtYXBzCgpBbm90aGVyIHdheSB0byBsb29rIGF0IGNvcnJlbGF0aW9uIGlmIHdlIGhhdmUgbWFueSB2YXJpYWJsZXMgaXMgdG8gc2hvdyB0aGUgc3RyZW5ndGggb2YgY29ycmVsYXRpb24gYmV0d2VlbiBwYWlycyBvZiB2YXJpYWJsZXMgdXNpbmcgYSBoZWF0bWFwLCB3aGVyZSB0aGUgaW50ZW5zaXR5IG9mIHRoZSBjb2xvciBpbmRpY2F0ZXMgdGhlIHN0cmVuZ3RoIG9mIHRoZSBjb3JyZWxhdGlvbiBiZXR3ZWVuIHR3byB2YXJpYWJsZXMuCgpMZXQncyBkbyB0aGlzIG5vdyBmb3IgdGhlIGRlbW9ncmFwaGljIHZhcmlhYmxlcyBmb3IgZWFjaCBhbmFseXNpcy4KClRoZSBgZ2djb3JycGxvdCgpYCBmdW5jdGlvbiBvZiB0aGUgYGdncGNvcnJwbG90YCBwYWNrYWdlIGlzIG9uZSB3YXkgdG8gY3JlYXRlIHN1Y2ggYSBoZWF0bWFwLgoKQXMgaW5wdXQgdG8gdGhlIHBsb3R0aW5nIGZ1bmN0aW9uLCB3ZSBmaXJzdCBuZWVkIHRvIGNhbGN1bGF0ZSB0aGUgY29ycmVsYXRpb24gdmFsdWVzLCB3aGljaCB3ZSB3aWxsIGRvIHVzaW5nIHRoZSBgY29yKClgIGZ1bmN0aW9uIG9mIHRoZSBgc3RhdHNgIHBhY2thZ2UuCgpUbyBsYWJlbCBvdXIgbGVnZW5kIHdpdGggdGhlIEdyZWVrIGxldHRlciAkXHJobyQsIHdlIHdpbGwgdXNlIHRoZSBiYXNlIGBleHByZXNzaW9uKClgIGZ1bmN0aW9uLCB3aGljaCB3aWxsIGNvbnZlcnQgdGhlIHdyaXR0ZW4gZm9ybSBvZiBgInJobyJgIHRvIHRoZSBHcmVlayBsZXR0ZXIuCgpgYGB7cn0KY29yX0RPTk9IVUVfZGVtIDwtIGNvcihET05PSFVFX0RGICU+JSBkcGx5cjo6c2VsZWN0KGNvbnRhaW5zKCJfeWVhcnMiKSkpCgpjb3JyX21hdF9ET05PSFVFIDwtIGdnY29ycnBsb3QoY29yX0RPTk9IVUVfZGVtLAogIHRsLmNleCA9IDYsCiAgaGMub3JkZXIgPSBUUlVFLAogIGNvbG9ycyA9IGMoCiAgICAicmVkIiwKICAgICJ3aGl0ZSIsCiAgICAicmVkIgogICksCiAgb3V0bGluZS5jb2xvciA9ICJ0cmFuc3BhcmVudCIsCiAgdGl0bGUgPSAiQ29ycmVsYXRpb24gTWF0cml4LCBBbmFseXNpcyAxIiwKICBsZWdlbmQudGl0bGUgPSBleHByZXNzaW9uKHJobykKKQoKCmNvcnJfbWF0X0RPTk9IVUUKCmNvcl9MT1RUX2RlbSA8LSBjb3IoTE9UVF9ERiAlPiUgZHBseXI6OnNlbGVjdChjb250YWlucygiX3llYXJzIikpKQoKY29ycl9tYXRfTE9UVCA8LSBnZ2NvcnJwbG90KGNvcl9MT1RUX2RlbSwKICB0bC5jZXggPSA2LAogIGhjLm9yZGVyID0gVFJVRSwKICBjb2xvcnMgPSBjKAogICAgInJlZCIsCiAgICAid2hpdGUiLAogICAgInJlZCIKICApLAogIG91dGxpbmUuY29sb3IgPSAidHJhbnNwYXJlbnQiLAogIHRpdGxlID0gIkNvcnJlbGF0aW9uIE1hdHJpeCwgQW5hbHlzaXMgMiIsCiAgbGVnZW5kLnRpdGxlID0gZXhwcmVzc2lvbihyaG8pCikKCmNvcnJfbWF0X0xPVFQKYGBgCldlIGNhbiBzZWUgdGhhdCBtYW55IG9mIHRoZSBkZW1vZ3JhcGhpYyB2YXJpYWJsZXMgYXJlIGhpZ2hseSBjb3JyZWxhdGVkIHdpdGggb25lIGFub3RoZXIsIGVpdGhlciBwb3NpdGl2ZWx5IG9yIG5lZ2F0aXZlbHkuIEluIHRoaXMgY2FzZSwgdGhlIHNpZ24gZG9lcyBub3QgbWF0dGVyLCBpbiB0ZXJtcyBvZiB0aGUgZWZmZWN0IHRoZSBjb2xsaW5lYXJpdHkgY291bGQgaGF2ZSBvbiBvdXIgbW9kZWxpbmcgcmVzdWx0cy4gCgoKVGhlIHByZXNlbmNlIG9mIGNvcnJlbGF0aW9uIGJldHdlZW4gdmFyaWFibGVzIHN1Z2dlc3RzIHRoYXQgd2UgbWlnaHQgaGF2ZSBtdWx0aWNvbGxpbmVhcml0eS4gSG93ZXZlciBpdCBkb2VzIG5vdCBuZWNlc3NhcmlseSBtZWFuIHRoYXQgd2UgZG8uIFNvIGhvdyBjYW4gd2UgYXNzZXNzIHRoaXM/CgoKIyMjICoqQ29lZmZpY2llbnQgZXN0aW1hdGUgaW5zdGFiaWxpdHkqKgoqKioKCk9uZSB3YXkgdG8gbG9vayBhdCB0aGUgcG9zc2libGUgaW5mbHVlbmNlIG9mIG11bHRpY29sbGluZWFyaXR5IGlzIHRvIGxvb2sgYXQgdGhlIHN0YWJpbGl0eSBvZiB0aGUgY29lZmZpY2llbnQgZXN0aW1hdGVzIHVuZGVyIHBlcnR1cmJhdGlvbnMgb2YgdGhlIGRhdGEuCgpXZSB3aWxsIGZvY3VzIG9uIHRoZSBgUlRDX0xBV2AgdmFyaWFibGUgY29lZmZpY2llbnQgZXN0aW1hdGUsIGFzIHRoaXMgaXMgb2YgcGFydGljdWxhciBpbnRlcmVzdCBpbiBvdXIgY2FzZS4KClRvIGRvIHNvIHdlIHdpbGwgcGVyZm9ybSBhIHByb2Nlc3MgY2FsbGVkIFtyZXNhbXBsaW5nXShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9SZXNhbXBsaW5nXyhzdGF0aXN0aWNzKSl7dGFyZ2V0PSJfYmxhbmsifS4gVGhpcyBpbnZvbHZlcyBwZXJmb3JtaW5nIG11bHRpcGxlIGl0ZXJhdGlvbnMgb2Ygb3VyIGFuYWx5c2lzLCBidXQgd2l0aCBvbmx5IGEgc3Vic2V0IG9yIHN1Yi1zYW1wbGUgb2Ygb3VyIG9yaWdpbmFsIGRhdGEuICBJbiBvdXIgY2FzZSB3ZSB3aWxsIHJlbW92ZSAqKm9uZSoqIG9ic2VydmF0aW9uIGFuZCBzZWUgaWYgdGhhdCBjaGFuZ2VzIG91ciBjb2VmZmljaWVudCBlc3RpbWF0ZSByZXN1bHRzLiAKCgpUbyBkbyB0aGlzIHdlIHdpbGwgdXNlIHNvbWUgZnVuY3Rpb25zIGluIHRoZSBgcnNhbXBsZWAgcGFja2FnZSB3aGljaCBpcyB2ZXJ5IHVzZWZ1bCBmb3Igc3BsaXR0aW5nIGRhdGEgaW4gdmFyaW91cyB3YXlzLgoKV2Ugd2lsbCB1c2UgdGhlIGBsb29fY3YoKWAgZnVuY3Rpb24gd2hpY2ggc3RhbmRzIGZvciAqKmxlYXZlIG9uZSBvdXQqKiBbY3Jvc3MgdmFsaWRhdGlvbl0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvQ3Jvc3MtdmFsaWRhdGlvbl8oc3RhdGlzdGljcykpe3RhcmdldD0iX2JsYW5rIn0uIFRoaXMgd2lsbCBhbGxvdyB1cyB0byBzcGxpdCBvdXIgZGF0YSBpbnRvIGV2ZXJ5IHBvc3NpYmxlIHN1YnNldCB3aGVyZSBhIHVuaXF1ZSBvYnNlcnZhdGlvbiBpcyBsZWZ0IG91dCBvZiB0aGUgZGF0YS4KClRoaXMgZnVuY3Rpb24gd2lsbCBob3dldmVyIG9ubHkgcHJlcGFyZSB0aGUgZGF0YSB0byBiZSBzcGxpdC4KClRvIGdldCB0aGUgcmVtYWluaW5nIGRhdGEgYWZ0ZXIgdGhlIHJlbW92YWwgb2YgdGhlIG9ic2VydmF0aW9uIHRoYXQgaXMgbGVmdCBvdXQgd2Ugd2lsbCB1c2UgYSBmdW5jdGlvbiBjYWxsZWQgYHRyYWluaW5nKClgLiBUaGVzZSBmdW5jdGlvbiBuYW1lcyBhcmlzZSBmcm9tIHRoZSBmYWN0IHRoYXQgdGhlc2UgZnVuY3Rpb25zIGFyZSBvZnRlbiB1c2VkIGZvciBpbiBtYWNoaW5lIGxlYXJuaW5nIGFwcGxpY2F0aW9ucyB3aGVyZSB0aGUgZGF0YSBpcyBzcGxpdCBiZXR3ZWVuIGEgbGFyZ2VyIHRyYWluaW5nIHNldCBhbmQgYSBzbWFsbGVyIHRlc3Rpbmcgc2V0LiBUaHVzIHdlIHdhbnQgdGhlIGxhcmdlciAkbi0xJCBzdWJzZXQsIGFzIG9wcG9zZWQgdG8gdGhlIHNpbmdsZSB2YWx1ZSB0aGF0IGlzIHJlbW92ZWQsICh3aGljaCB3ZSBjb3VsZCBnZXQgd2l0aCB0aGUgYHRlc3RpbmcoKWAgZnVuY3Rpb24pLgoKKioqCjxkZXRhaWxzPiA8c3VtbWFyeT4gQ2xpY2sgaGVyZSB0byBzZWUgYW4gZXhhbXBsZSBvZiBob3cgdGhpcyB3b3Jrcy4gPC9zdW1tYXJ5PgoKRmlyc3Qgd2Ugd2lsbCBtYWtlIGEgdG95IGRhdGFzZXQgdGhhdCBpcyB2ZXJ5IHNpbXBsZSBjYWxsZWQgYHRlc3RgIHVzaW5nIHRoZSBgdGliYmxlKClgIGZ1bmN0aW9uIG9mIHRoZSBgdGliYmxlYCBwYWNrYWdlOgpgYGB7cn0KdGVzdCA8LSB0aWJibGU6OnRpYmJsZSh4ID0gYygxLCAyLCAzKSkKdGVzdApgYGAKCk5vdyB3ZSB3aWxsIHVzZSB0aGUgYGxvb19jdigpYCB0byBjcmVhdGUgbGVhdmUgb25lIG91dCBzcGxpdHM6CmBgYHtyfQp0ZXN0X3NhbXBsZXMgPC0gdGVzdCAlPiUgcnNhbXBsZTo6bG9vX2N2KCkKdGVzdF9zYW1wbGVzCmBgYAoKV2UgY2FuIHRha2UgYSBsb29rIGF0IGEgc2luZ2xlIHNwbGl0IG9mIHRoZSBkYXRhIHVzaW5nIHRoZSBgcHVsbCgpYCBmdW5jdGlvbjoKCmBgYHtyfQpwdWxsKHRlc3Rfc2FtcGxlcywgc3BsaXRzKQpgYGAKCkhlcmUgeW91IGNhbiBzZWUgdGhhdCAyIHZhbHVlcyBhcmUgaW50ZW5kZWQgZm9yIHRoZSB0cmFpbmluZyBzZXQgKGFsc28gY2FsbGVkIEFuYWx5c2lzIHNldCksIDEgdmFsdWUgaXMgaW50ZW5kZWQgZm9yIHRoZSB0ZXN0aW5nIHNldCAoYWxzbyBjYWxsZWQgQXNzZXNzbWVudCBzZXQpLCBhbmQgMyB2YWx1ZXMgd2VyZSBwcmVzZW50IGluaXRpYWxseS4KCk5vdyB3ZSB3aWxsIHVzZSB0aGUgYHRyYWluaW5nKClgIGZ1bmN0aW9uIHRvIGdldCB0aGUgZGF0YSB3aXRob3V0IHRoZSBvYnNlcnZhdGlvbiB0aGF0IGlzIHNldCBhc2lkZS4gSGVyZSBpcyB0aGUgZGF0YSBmb3IgdGhlIGZpcnN0IHN1YnNldDoKYGBge3J9CnJzYW1wbGU6OnRyYWluaW5nKHB1bGwodGVzdF9zYW1wbGVzLCBzcGxpdHMpW1sxXV0pCmBgYAoKTm93IHdlIHdpbGwgdXNlIHRoZSBgbWFwKClgIGZ1bmN0aW9uIG9mIGBwdXJycmAgdG8gZ2V0IGFsbCBwb3NzaWJsZSBgdHJhaW5pbmdgIHN1YnNldCBvZiB0aGUgZGF0YS4KYGBge3J9Cgp0ZXN0X3N1YnNldHMgPC0gbWFwKHB1bGwodGVzdF9zYW1wbGVzLCBzcGxpdHMpLCB0cmFpbmluZykKdGVzdF9zdWJzZXRzCmBgYAoKV2UgY2FuIHNlZSB0aGF0IHRoZXJlIGFyZSAzIHBvc3NpYmxlIHN1YnNldHMgdGhhdCBsZWF2ZSBvbmUgdmFsdWUgb3V0LiBBbGwgMyBwb3NzaWJsZSBzdWJzZXRzIGFyZSBjcmVhdGVkIHVzaW5nIHRoaXMgbWV0aG9kLiBUaGlzIG1ldGhvZCB3aWxsIGFsd2F5cyBjcmVhdGUgdGhlIHNhbWUgbnVtYmVyIG9mIHN1YnNldHMgYXMgdGhlcmUgYXJlIHVuaXF1ZSB2YWx1ZXMgb3Igcm93cyBpbiB0aGUgZGF0YS4KCjwvZGV0YWlscz4KKioqCgpOb3cgd2Ugd2lsbCB1c2UgdGhpcyBtZXRob2Qgd2l0aCB0aGUgZGF0YSBmcm9tIG91ciBEb25vaHVlLWxpa2UgYW5hbHlzaXMsIGFuZCBzaW5jZSB0aGlzIGRhdGEgaGFzIGByIGRpbShkX3BhbmVsX0RPTk9IVUUpWzFdYCByb3dzLCBgciBkaW0oZF9wYW5lbF9ET05PSFVFKVsxXWAgc3Vic2V0cyB3aWxsIGJlIGNyZWF0ZWQgdGhhdCBsZWF2ZSBvdXQgb25lIHJvdy4gVGhlIGlkZWEgaXMgdG8gZml0IG91ciBwYW5lbCByZWdyZXNzaW9uIG1vZGVsIG9uIGVhY2ggc3Vic2V0IG9mIHRoZSBkYXRhLCBhbmQgdGhlbiBleGFtaW5lIGhvdyB0aGUgY29lZmZpY2llbnQgZXN0aW1hdGVzIGZyb20gZWFjaCBvZiB0aGVzZSBtb2RlbCBmaXRzIHZhcnkgYXMgdGhlIHNhbXBsZSBjaGFuZ2VzIHNsaWdodGx5LiBXaXRoIGNvbGxpbmVhciBwcmVkaWN0b3JzLCB3ZSBleHBlY3QgdGhhdCBvdXIgY29lZmZpY2llbnQgZXN0aW1hdGVzIG1heSBiZSB1bnN0YWJsZSBhbmQgc3ViamVjdCB0byBjaGFuZ2UgdW5kZXIgZXZlbiBzbWFsbCBwZXJ0dXJiYXRpb25zIG9mIHRoZSBkYXRhLgoKRmlyc3Qgd2Ugd2lsbCBjcmVhdGUgdGhlIHNwbGl0cyB1c2luZyB0aGUgYGxvb19jdigpYCBmdW5jdGlvbi4gVGhpcyB3aWxsIHVsdGltYXRlbHkgbWFrZSBhbGwgdW5pcXVlIHBvc3NpYmxlIHN1YnNldHMgbGVhdmluZyBvbmUgc2FtcGxlIG91dCAtIGhvd2V2ZXIgdGhlIGNob2ljZSBvZiB3aGljaCBzYW1wbGUgaXMgbGVmdCBmb3Igd2hpY2ggaXRlcmF0aW9uIGlzIGEgcmFuZG9tIHByb2Nlc3MgKG1heWJlIHRoZSAxMnRoIGlzIGxlZnQgb3V0IGZpcnN0IHRoZW4gdGhlIDNyZCBldGMuKS4gIFRodXMsIHdlIHVzZSB0aGUgYHNldC5zZWVkKClgIGZ1bmN0aW9uIHRvIGVuc3VyZSByZXByb2R1Y2liaWxpdHkgb2YgdGhlIHJlc3VsdHMgd2hlbiB3ZSB0YWtlIGEgbG9vayBhdCB0aGUgZmlyc3Qgc3Vic2V0LCBzbyB0aGF0IGl0IHdpbGwgYmUgdGhlIHNhbWUgZm9yIGV2ZXJ5b25lIHdobyBsb29rcyBhdCB0aGlzIGNhc2Ugc3R1ZHkuIFlvdSBjYW4gY2hhbmdlIHRoZSBzZWVkIHRvIHBsYXkgYXJvdW5kIHdpdGggdGhpcy4KCmBgYHtyfQpzZXQuc2VlZCgxMjQpCkRPTk9IVUVfc3BsaXRzIDwtIGRfcGFuZWxfRE9OT0hVRSAlPiUgbG9vX2N2KCkKRE9OT0hVRV9zcGxpdHMKYGBgCgpOb3cgd2Ugd2lsbCB1c2UgdGhlIGB0cmFpbmluZygpYCBmdW5jdGlvbiB0byBzZWxlY3QgdGhlIHJlbWFpbmluZyBkYXRhIHdpdGhvdXQgdGhlIHZhbHVlIHRoYXQgd2FzIHJlbW92ZWQgZm9yIGVhY2ggc3BsaXQ6CgpgYGB7cn0KIyBUbyBnZXQgYWxsIHRoZSBkYXRhIHN1YnNldHMKRE9OT0hVRV9zdWJzZXRzIDwtIG1hcChwdWxsKERPTk9IVUVfc3BsaXRzLCBzcGxpdHMpLCB0cmFpbmluZykKCmdsaW1wc2UoRE9OT0hVRV9zdWJzZXRzW1sxXV0pCmxlbmd0aChET05PSFVFX3N1YnNldHMpCmBgYAoKQXMgZXhwZWN0ZWQgdGhlIGZpcnN0IHN1YnNldCBoYXMgMTM5NCByb3dzIGFuZCB0aGVyZSBhcmUgMTM5NSBzdWJzZXRzLgoKTGV0J3Mgc2VlIHdoYXQgb2JzZXJ2YXRpb24gd2FzIGxlZnQgb3V0IGluIHRoZSBmaXJzdCBzdWJzZXQ6CgpgYGB7cn0KZF9wYW5lbF9ET05PSFVFICU+JQogIGZpbHRlcighcm93bmFtZXMoZF9wYW5lbF9ET05PSFVFKSAlaW4lIHJvd25hbWVzKERPTk9IVUVfc3Vic2V0c1tbMV1dKSkKCiMgQW5vdGhlciB3YXkgdG8gY2hlY2sgaXMgdG8gdXNlOgpET05PSFVFX3JlbW92ZWQgPC0gbWFwKHB1bGwoRE9OT0hVRV9zcGxpdHMsIHNwbGl0cyksIHRlc3RpbmcpCgpET05PSFVFX3JlbW92ZWRbWzFdXQpgYGAKCkl0IGxvb2tzIGxpa2UgdGhlIFRleGFzIGRhdGEgZnJvbSAxOTg4IHdhcyByZW1vdmVkIGZyb20gdGhlIGZpcnN0IHNwbGl0LiBBZ2FpbiwgaWYgeW91IHRyeSBkaWZmZXJlbnQgc2VlZHMgeW91IHdpbGwgc2VlIGEgZGlmZmVyZW50IHNhbXBsZSByZW1vdmVkIGZyb20gdGhlIGZpcnN0IHNwbGl0LgoKT0ssIHNvIG5vdyBsZXQncyBmaXQgb3VyIHBhbmVsIHJlZ3Jlc3Npb24gb24gdGhlIGZpcnN0IHN1YnNldCBvZiBkYXRhIGxpa2Ugd2UgZGlkIHByZXZpb3VzbHkuIE5vdGUgdGhhdCB0aGlzIGNhdXNlcyBvdXIgZGF0YSB0byBiZSBhbiAqKnVuYmFsYW5jZWQqKiBwYW5lbC4gVGhpcyBkb2VzIG5vdCByZXF1aXJlIGFueSBhZGp1c3RtZW50IHRvIHRoZSBjb2RlIHRvIG1vZGVsIHRoZSBkYXRhLCBidXQgeW91IHdpbGwgbm90aWNlIHRoYXQgdGhlIG91dHB1dCB3aWxsIG5vdyBzYXkgInVuYmFsYW5jZWQiLgoKYGBge3J9CnN1YnNldF8xX3Jlc3VsdCA8LSBwbG0oVmlvbF9jcmltZV9yYXRlXzFrX2xvZyB+ClJUQ19MQVcgKwogIFdoaXRlX01hbGVfMTVfdG9fMTlfeWVhcnMgKwogIFdoaXRlX01hbGVfMjBfdG9fMzlfeWVhcnMgKwogIEJsYWNrX01hbGVfMTVfdG9fMTlfeWVhcnMgKwogIEJsYWNrX01hbGVfMjBfdG9fMzlfeWVhcnMgKwogIE90aGVyX01hbGVfMTVfdG9fMTlfeWVhcnMgKwogIE90aGVyX01hbGVfMjBfdG9fMzlfeWVhcnMgKwogIFVuZW1wbG95bWVudF9yYXRlICsKICBQb3ZlcnR5X3JhdGUgKwogIFBvcHVsYXRpb25fbG9nICsKICBwb2xpY2VfcGVyXzEwMGtfbGFnLApkYXRhID0gRE9OT0hVRV9zdWJzZXRzW1sxXV0sCmluZGV4ID0gYygiU1RBVEUiLCAiWUVBUiIpLAptb2RlbCA9ICJ3aXRoaW4iLAplZmZlY3QgPSAidHdvd2F5cyIKKQpzdW1tYXJ5KHN1YnNldF8xX3Jlc3VsdCkKYGBgCgpJbmRlZWQsIHdlIGNhbiBzZWUgdGhhdCBub3cgd2UgaGF2ZSBhbiB1bmJhbGFuY2VkIHBhbmVsIHdpdGggTiA9IDEzOTQgb2JzZXJ2YXRpb25zIGluc3RlYWQgb2YgMTM5NSwgYXMgZXhwZWN0ZWQuCgpOb3cgdGhhdCB3ZSBoYXZlIG91ciBzdWJzZXRzLCB3ZSB3YW50IHRvIHdyaXRlIGEgZnVuY3Rpb24gdG8gZml0IGEgcGFuZWwgcmVncmVzc2lvbiB1c2luZyBgcGxtKClgb24gZWFjaCBzdWJzZXQuIFNlZSBbdGhpcyBjYXNlIHN0dWR5XShodHRwczovL29wZW5jYXNlc3R1ZGllcy5naXRodWIuaW8vb2NzLWJsb29tYmVyZy12YXBpbmctY2FzZS1zdHVkeS8pe3RhcmdldD0iX2JsYW5rIn0gZm9yIG1vcmUgaW5mb3JtYXRpb24gb24gd3JpdGluZyBmdW5jdGlvbnMuCgpgYGB7cn0KZml0X25sc19vbl9ib290c3RyYXBfRE9OT0hVRSA8LSBmdW5jdGlvbihzdWJzZXQpIHsKICBwbG0oVmlvbF9jcmltZV9yYXRlXzFrX2xvZyB+CiAgUlRDX0xBVyArCiAgICBXaGl0ZV9NYWxlXzE1X3RvXzE5X3llYXJzICsKICAgIFdoaXRlX01hbGVfMjBfdG9fMzlfeWVhcnMgKwogICAgQmxhY2tfTWFsZV8xNV90b18xOV95ZWFycyArCiAgICBCbGFja19NYWxlXzIwX3RvXzM5X3llYXJzICsKICAgIE90aGVyX01hbGVfMTVfdG9fMTlfeWVhcnMgKwogICAgT3RoZXJfTWFsZV8yMF90b18zOV95ZWFycyArCiAgICBVbmVtcGxveW1lbnRfcmF0ZSArCiAgICBQb3ZlcnR5X3JhdGUgKwogICAgUG9wdWxhdGlvbl9sb2cgKwogICAgcG9saWNlX3Blcl8xMDBrX2xhZywKICBkYXRhID0gZGF0YS5mcmFtZShzdWJzZXQpLAogIGluZGV4ID0gYygiU1RBVEUiLCAiWUVBUiIpLAogIG1vZGVsID0gIndpdGhpbiIsCiAgZWZmZWN0ID0gInR3b3dheXMiCiAgKQp9CmBgYAoKTm93IHdlIGNhbiBhcHBseSB0aGlzIGZ1bmN0aW9uIHRvIGVhY2ggb2Ygb3VyIHN1YnNldHMgc2ltdWx0YW5lb3VzbHkgdXNpbmcgdGhlIGBtYXAoKWAgZnVuY3Rpb24gb2YgdGhlIGBwdXJycmAgcGFja2FnZS4gIAoKYGBge3IsIGV2YWwgPSBGQUxTRX0KCnN1YnNldHNfbW9kZWxzX0RPTk9IVUUgPC0gbWFwKERPTk9IVUVfc3Vic2V0cywgZml0X25sc19vbl9ib290c3RyYXBfRE9OT0hVRSkKCnN1YnNldHNfbW9kZWxzX0RPTk9IVUUgPC0gc3Vic2V0c19tb2RlbHNfRE9OT0hVRSAlPiUKICBtYXAodGlkeSkKYGBgCgpXZSB3aWxsIHNhdmUgdGhpcyBkYXRhIGluIG91ciBgZGF0YWAgZGlyZWN0b3J5IHdpdGhpbiBhIHN1YmRpcmVjdG9yeSBjYWxsZWQgYHdyYW5nbGVkYCBsaWtlIHNvOgoKYGBge3IsIGV2YWwgPSBGQUxTRX0Kc2F2ZShzdWJzZXRzX21vZGVsc19ET05PSFVFLAogIGZpbGUgPSBoZXJlOjpoZXJlKCJkYXRhIiwgIndyYW5nbGVkIiwgIkRPTk9IVUVfc2ltdWxhdGlvbnMucmRhIikKKQpgYGAKCklmIHlvdSBoYXZlIHRyb3VibGUgd2l0aCB0aGUgcHJldmlvdXMgc3RlcCB5b3UgY2FuIGxvYWQgdGhlIGRhdGEgbGlrZSBzbzoKCmBgYHtyfQpsb2FkKGhlcmU6OmhlcmUoImRhdGEiLCAid3JhbmdsZWQiLCAiRE9OT0hVRV9zaW11bGF0aW9ucy5yZGEiKSkKYGBgCgoKCkdyZWF0ISBOb3cgd2Ugd2FudCB0byBkbyB0aGUgc2FtZSB0aGluZyBmb3IgdGhlIE11c3RhcmQgYW5kIExvdHQgZGF0YS4KCmBgYHtyfQpzZXQuc2VlZCgxMjQpCkxPVFRfc3BsaXRzIDwtIGRfcGFuZWxfTE9UVCAlPiUgbG9vX2N2KCkKCiMgVG8gZ2V0IGFsbCB0aGUgZGF0YSBzdWJzZXRzOgpMT1RUX3N1YnNldHMgPC0gbWFwKHB1bGwoTE9UVF9zcGxpdHMsIHNwbGl0cyksIHRyYWluaW5nKQpgYGAKCgpXZSBuZWVkIHRvIGNyZWF0ZSBhIGRpZmZlcmVudCBmdW5jdGlvbiB0byBmaXQgdGhlIGRhdGEgdG8gYWNjb3VudCBmb3IgdGhlIGxhcmdlciBudW1iZXIgb2YgZGVtb2dyYXBoaWMgdmFyaWFibGVzLiBXZSB3aWxsIHVzZSB0aGUgZm9ybXVsYSB0aGF0IHdlIG1hZGUgcHJldmlvdXNseS4KCmBgYHtyfQpmaXRfbmxzX29uX2Jvb3RzdHJhcF9MT1RUIDwtIGZ1bmN0aW9uKHNwbGl0KSB7CiAgcGxtKExPVFRfZm1sYSwKICAgIGRhdGEgPSBkYXRhLmZyYW1lKHNwbGl0KSwKICAgIGluZGV4ID0gYygiU1RBVEUiLCAiWUVBUiIpLAogICAgbW9kZWwgPSAid2l0aGluIiwKICAgIGVmZmVjdCA9ICJ0d293YXlzIgogICkKfQpgYGAKCgoKYGBge3IsIGV2YWwgPSBGQUxTRX0Kc3Vic2V0c19tb2RlbHNfTE9UVCA8LSBtYXAoTE9UVF9zdWJzZXRzLCBmaXRfbmxzX29uX2Jvb3RzdHJhcF9MT1RUKQoKc3Vic2V0c19tb2RlbHNfTE9UVCA8LSBzdWJzZXRzX21vZGVsc19MT1RUICU+JQogIG1hcCh0aWR5KQpgYGAKCkFnYWluIHdlIHdpbGwgc2F2ZSB0aGlzIGluIHRoZSBgd3JhbmdsZWRgIHN1YmRpcmVjdG9yeSBvZiBvdXIgYGRhdGFgIGRpcmVjdG9yeToKCmBgYHtyLCBldmFsID0gRkFMU0V9CnNhdmUoc3Vic2V0c19tb2RlbHNfTE9UVCwKICBmaWxlID0gaGVyZTo6aGVyZSgiZGF0YSIsICJ3cmFuZ2xlZCIsICJMT1RUX3NpbXVsYXRpb25zLnJkYSIpCikKYGBgCgpJZiB5b3UgaGF2ZSB0cm91YmxlIHdpdGggdGhlIHByZXZpb3VzIHN0ZXBzIHlvdSBjYW4gbG9hZCB0aGlzIGRhdGEgbGlrZSBzbzoKCmBgYHtyfQpsb2FkKGhlcmU6OmhlcmUoImRhdGEiLCAid3JhbmdsZWQiLCAiTE9UVF9zaW11bGF0aW9ucy5yZGEiKSkKYGBgCgoKTm93IHdlIHdpbGwgY29tYmluZSB0aGUgb3V0cHV0IHNvIHRoYXQgd2UgY2FuIG1ha2UgYSBwbG90IHRvIHZpc3VhbGl6ZSB0aGUgcmVzdWx0cyB0aGF0IHdlIG9idGFpbmVkLCBpLmUuLCB0byBsb29rIGF0IGhvdyB0aGUgdmFyaWF0aW9uIGluIG91ciBjb2VmZmljaWVudCBlc3RpbWF0ZXMgYWNyb3NzIGRhdGEgc3Vic2V0cyB2YXJpZXMgYmV0d2VlbiB0aGUgRG9ub2h1ZSBhbmQgdGhlIExvdHQgYW5kIE11c3RhcmQgbW9kZWxzLiBGaXJzdCBsZXQncyBuYW1lIGVhY2ggc3Vic2V0IHRoYXQgd2UgY3JlYXRlZC4KCmBgYHtyfQpuYW1lcyhzdWJzZXRzX21vZGVsc19ET05PSFVFKSA8LSBwYXN0ZTAoIkRPTk9IVUVfIiwgc2VxX2xlbihsZW5ndGgoc3Vic2V0c19tb2RlbHNfRE9OT0hVRSkpKQoKbmFtZXMoc3Vic2V0c19tb2RlbHNfTE9UVCkgPC0KICBwYXN0ZTAoIkxPVFRfIiwgMTpsZW5ndGgoc3Vic2V0c19tb2RlbHNfTE9UVCkpCmBgYAoKTm93IHdlIGNhbiBjb21iaW5lIHRoZSB0aWJibGVzIHdpdGhpbiB0aGUgbGlzdCBvZiB0aWJibGVzIGZvciB0aGUgYHN1YnNldHNfbW9kZWxzX0RPTk9IVUVgIGFuZCBgc3Vic2V0c19tb2RlbHNfTE9UVGAgZGF0YS4KClRvIGRvIHRoaXMgd2Ugd2lsbCB1c2UgdGhlIGBiaW5kX3Jvd3MoKWAgZnVuY3Rpb24gb2YgdGhlIGBkcGx5cmAgcGFja2FnZSB3aXRoIHRoZSBgLmlkID0gIklEImAgYXJndW1lbnQsIHdoaWNoIHdpbGwgY3JlYXRlIGEgbmV3IHZhcmlhYmxlIGNhbGxlZCBgSURgIHRoYXQgd2lsbCBsaXN0IHRoZSBuYW1lIG9mIHRoZSB0aWJibGUgdGhlIGRhdGEgY2FtZSBmcm9tLgoKVGhlbiB3ZSB3aWxsIGNvbWJpbmUgdGhlIGRhdGEgZnJvbSBib3RoIHRoZSBEb25vaHVlIGFuZCBMb3R0IHNpbXVsYXRpb25zLgoKYGBge3J9CgpzaW11bGF0aW9uc19ET05PSFVFIDwtIHN1YnNldHNfbW9kZWxzX0RPTk9IVUUgJT4lCiAgYmluZF9yb3dzKC5pZCA9ICJJRCIpICU+JQogIG11dGF0ZShBbmFseXNpcyA9ICJBbmFseXNpcyAxIikKCnNpbXVsYXRpb25zX0xPVFQgPC0gc3Vic2V0c19tb2RlbHNfTE9UVCAlPiUKICBiaW5kX3Jvd3MoLmlkID0gIklEIikgJT4lCiAgbXV0YXRlKEFuYWx5c2lzID0gIkFuYWx5c2lzIDIiKQoKc2ltdWxhdGlvbnMgPC0gYmluZF9yb3dzKAogIHNpbXVsYXRpb25zX0RPTk9IVUUsCiAgc2ltdWxhdGlvbnNfTE9UVAopCgpoZWFkKHNpbXVsYXRpb25zKQp0YWlsKHNpbXVsYXRpb25zKQpgYGAKCk5vdyB3ZSB3aWxsIG1ha2UgYSBzZXQgb2YgcGFyYWxsZWwgYm94cGxvdHMgdXNpbmcgdGhlIGBnZW9tX2JveHBsb3QoKWAgZnVuY3Rpb24gb2YgdGhlIGNvZWZmaWNpZW50IGVzdGltYXRlcyBvZiB0aGUgYFJUQ19MQVdUUlVFYCB2YXJpYWJsZSBmb3IgZWFjaCBzaW11bGF0aW9uLgoKU2luY2UgdGhlcmUgYXJlIG1hbnkgdmFyaWFibGVzIGluIGJvdGggYW5hbHlzZXMsIHdlIHdpbGwgdXNlIHRoZSBgZmFjZXRfZ3JpZCgpYCBmdW5jdGlvbiBvZiB0aGUgYGdncGxvdDJgIHBhY2thZ2UgdG8gYWxsb3cgdXMgdG8gc2VwYXJhdGUgdGhlIGRhdGEgZm9yIGVhY2ggYW5hbHlzaXMgaW50byBzdWJwbG90cy4gVGhlIGFyZ3VtZW50IGBzY2FsZSA9ICJmcmVlX3giYCBhbmQgYGRyb3AgPSBUUlVFYCBhbGxvdyB1cyB0byBvbmx5IGluY2x1ZGUgdGhlIHZhcmlhYmxlcyB0aGF0IHdlcmUgcHJlc2VudCBpbiBBbmFseXNpcyAxLCBhcyBvcHBvc2VkIHRvIGVtcHR5IHNwb3RzIGZvciB0aGUgdmFyaWFibGVzIHRoYXQgd2VyZSBpbiBBbmFseXNpcyAyIGJ1dCBub3QgaW4gQW5hbHlzaXMgMS4gVGhlIGBzcGFjZSA9ICJmcmVlImAgYXJndW1lbnQgcmVtb3ZlcyB0aGUgZXh0cmEgc3BhY2UgZnJvbSB0aGUgZHJvcHBlZCB2YXJpYWJsZXMuIAoKIyMjIyB7LnF1ZXN0aW9uX2Jsb2NrfQo8Yj48dT4gUXVlc3Rpb24gT3Bwb3J0dW5pdHkgPC91PjwvYj4KCldoYXQgaGFwcGVucyBpZiB5b3UgZG9uJ3QgdXNlIHRoZSBgZHJvcCA9IFRSVUVgIGFyZ3VtZW50IG9yIHRoZSBgc3BhY2UgPSAiZnJlZSJgIGFyZ3VtZW50PyAKCiMjIyMKCmBgYHtyfQpzaW11bGF0aW9uX3Bsb3QgPC0gc2ltdWxhdGlvbnMgJT4lCiAgZ2dwbG90KGFlcyh4ID0gdGVybSwgeSA9IGVzdGltYXRlKSkgKwogIGdlb21fYm94cGxvdCgpICsKICBmYWNldF9ncmlkKC4gfiBBbmFseXNpcywgc2NhbGUgPSAiZnJlZV94Iiwgc3BhY2UgPSAiZnJlZSIsIGRyb3AgPSBUUlVFKSArCiAgbGFicygKICAgIHRpdGxlID0gIkNvZWZmaWNpZW50IGVzdGltYXRlcyIsCiAgICBzdWJ0aXRsZSA9ICJFc3RpbWF0ZXMgYWNyb3NzIGxlYXZlLW9uZS1vdXQgYW5hbHlzZXMiLAogICAgeCA9ICJUZXJtIiwKICAgIHkgPSAiQ29lZmZpY2llbnQiLAogICAgY2FwdGlvbiA9ICJSZXN1bHRzIGZyb20gc2ltdWxhdGlvbnMiCiAgKSArCiAgdGhlbWVfbGluZWRyYXcoKSArCiAgdGhlbWUoCiAgICBheGlzLnRpdGxlLnggPSBlbGVtZW50X2JsYW5rKCksCiAgICBheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChhbmdsZSA9IDcwLCBoanVzdCA9IDEpLAogICAgc3RyaXAudGV4dC54ID0gZWxlbWVudF90ZXh0KHNpemUgPSAxNCwgZmFjZSA9ICJib2xkIikKICApCgpzaW11bGF0aW9uX3Bsb3QKYGBgCgpIZXJlLCB3ZSBjYW4gc3RhcnQgdG8gc2VlIHRoYXQgdGhlcmUgaXMgYSBiaXQgbW9yZSB2YXJpYWJpbGl0eSBpbiB0aGUgY29lZmZpY2llbnQgZXN0aW1hdGVzIGZyb20gdGhlIGxlYXZlLW9uZS1vdXQgcmVzdWx0cyBmb3IgQW5hbHlzaXMgMi4KCkZvciBvdXIgZGlzcGxheSBwdXJwb3Nlcywgd2Ugd291bGQgbGlrZSB0byBvcmRlciB0aGUgY292YXJpYXRlcyBzbyB0aGF0IHRoZXkgYXJlIGRpc3BsYXllZCBzaW1pbGFybHkgYWNyb3NzIHRoZSB0d28gcGFuZWxzLiBUaGlzIHdpbGwgYWxsb3cgdXMgdG8gYmV0dGVyIG9ic2VydmUgaG93IHRoZSBjb2VmZmljaWVudHMgb2YgdGhlIHNhbWUgY292YXJpYXRlIGJlaGF2ZSBpbiB0aGUgZGlmZmVyZW50IGFuYWx5c2VzLiBXZSB1c2UgdGhlIGBtdXRhdGVgIGZ1bmN0aW9uIHRvIGNvbnZlcnQgdGhlIGB0ZXJtYCB2YXJpYWJsZSB0byBhIGZhY3Rvciwgd2hlcmUgd2UgYXNzaWduIHRoZSBub24tZGVtb2dyYXBoaWMgdmFyaWFibGVzIHRvIGJlIHRoZSBmaXJzdCBsZXZlbHMsIHdpdGggdGhlIHNvcnRlZCAoaW4gYWxwaGFiZXRpY2FsIG9yZGVyIHVzaW5nIHRoZSBiYXNlIGBzb3J0KClgIGZ1bmN0aW9uKSBkZW1vZ3JhcGhpYyB2YXJpYWJsZXMgY29taW5nIGFmdGVyd2FyZHMuCgpZb3UgbWlnaHQgbm90aWNlIHRoYXQgdGhlIG5hbWVzIG9mIHRoZSBkZW1vZ3JhcGhpYyB2YXJpYWJsZSB2YWx1ZXMgaW4gdGhlIGB0ZXJtYCB2YXJpYWJsZSBvZiB0aGUgYHNpbXVsYXRpb25zYCBkYXRhIGFsbCBoYXZlIHRoZSB3b3JkICJ5ZWFycyIuIFdlIGNhbiB1c2UgdGhlIGBzdHJfc3Vic2V0KClgIGZ1bmN0aW9uIG9mIHRoZSBgc3RyaW5ncmAgcGFja2FnZSB0byBzZWxlY3QganVzdCB0aGUgZGVtb2dyYXBoaWMgdmFyaWFibGVzIGJhc2VkIG9uIHRoZSB3b3JkICJ5ZWFycy4gSW4gY29udHJhc3QsIHdlIGNhbiB1c2UgdGhlIGBuZWdhdGUgPSBUUlVFYCBhcmd1bWVudCB0byBzZWxlY3QgYWxsIHRoZSBvdGhlciB2YXJpYWJsZXMsIHRoZSBub24tZGVtb2dyYXBoaWMgdmFyaWFibGUgdmFsdWVzLiBXZSBjYW4gdXNlIHRoZSBgdW5pcXVlKClgIGJhc2UgZnVuY3Rpb24gdG8gZ3JhYiBqdXN0IHRoZSB1bmlxdWUgdmFsdWVzIG9mIHRoZSBgdGVybWAgdmFyaWFibGUuCgpgYGB7cn0Kc3RyX3N1YnNldCh1bmlxdWUocHVsbChzaW11bGF0aW9ucywgdGVybSkpLCAieWVhcnMiLCBuZWdhdGUgPSBUUlVFKQpgYGAKCmBgYHtyfQpzdHJfc3Vic2V0KHVuaXF1ZShwdWxsKHNpbXVsYXRpb25zLCB0ZXJtKSksICJ5ZWFycyIpCmBgYAoKTm93IHdlIGNhbiBjcmVhdGUgdGhlIG9yZGVyIG9mIHRoZSB2YWx1ZXMgb2YgdGhlIGB0ZXJtYCB2YXJpYWJsZSB1c2luZyB0aGUgYGZhY3RvcmAgYXJndW1lbnQgYW5kIHRoZSBgbGV2ZWxzYCBhcmd1bWVudC4KCmBgYHtyfQoKc2ltdWxhdGlvbnMgPC0gc2ltdWxhdGlvbnMgJT4lCiAgbXV0YXRlKHRlcm0gPSBmYWN0b3IodGVybSwKICAgIGxldmVscyA9IGMoCiAgICAgIHN0cl9zdWJzZXQodW5pcXVlKHB1bGwoc2ltdWxhdGlvbnMsIHRlcm0pKSwgInllYXJzIiwgbmVnYXRlID0gVFJVRSksCiAgICAgIHNvcnQoc3RyX3N1YnNldCh1bmlxdWUocHVsbChzaW11bGF0aW9ucywgdGVybSkpLCAieWVhcnMiKSkKICAgICkKICApKQoKbGV2ZWxzKHB1bGwoc2ltdWxhdGlvbnMsIHRlcm0pKQpgYGAKCkxvb2tzIGdvb2QhCgoKTm93IHdlIGp1c3QgbmVlZCB0byBydW4gdGhlIHNhbWUgY29kZSBhZ2FpbiB0byBjcmVhdGUgdGhlIHBsb3QsIGJ1dCBub3cgdGhlIG9yZGVyIG9mIHRoZSB4IGF4aXMgdmFsdWVzIHdpbGwgYmUgZGlmZmVyZW50LiAKCmBgYHtyfQoKc2ltdWxhdGlvbl9wbG90IDwtIHNpbXVsYXRpb25zICU+JQogIGdncGxvdChhZXMoeCA9IHRlcm0sIHkgPSBlc3RpbWF0ZSkpICsKICBnZW9tX2JveHBsb3QoKSArCiAgZmFjZXRfZ3JpZCguIH4gQW5hbHlzaXMsIHNjYWxlID0gImZyZWVfeCIsIHNwYWNlID0gImZyZWUiLCBkcm9wID0gVFJVRSkgKwogIGxhYnMoCiAgICB0aXRsZSA9ICJDb2VmZmljaWVudCBlc3RpbWF0ZXMiLAogICAgc3VidGl0bGUgPSAiRXN0aW1hdGVzIGFjcm9zcyBsZWF2ZS1vbmUtb3V0IGFuYWx5c2VzIiwKICAgIHggPSAiVGVybSIsCiAgICB5ID0gIkNvZWZmaWNpZW50IiwKICAgIGNhcHRpb24gPSAiUmVzdWx0cyBmcm9tIHNpbXVsYXRpb25zIgogICkgKwogIHRoZW1lX2xpbmVkcmF3KCkgKwogIHRoZW1lKAogICAgYXhpcy50aXRsZS54ID0gZWxlbWVudF9ibGFuaygpLAogICAgYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoYW5nbGUgPSA3MCwgaGp1c3QgPSAxKSwKICAgIHN0cmlwLnRleHQueCA9IGVsZW1lbnRfdGV4dChzaXplID0gMTQsIGZhY2UgPSAiYm9sZCIpCiAgKQoKc2ltdWxhdGlvbl9wbG90CmBgYAoKCldlIGNhbiBzZWUgdGhhdCB0aGUgcmFuZ2Ugb2YgY29lZmZpY2llbnQgZXN0aW1hdGVzIHdoZW4gb25seSBvbmUgb2JzZXJ2YXRpb24gaXMgcmVtb3ZlZCBpcyBtdWNoIGxhcmdlciBpbiBBbmFseXNpcyAyIGZvciBuZWFybHkgYWxsIHZhcmlhYmxlcywgYnV0IHBhcnRpY3VsYXJseSBmb3IgbWFueSBvZiB0aGUgYWRkaXRpb25hbCBkZW1vZ3JhcGhpYyB2YXJpYWJsZXMuCgpMZXQncyBtYWtlIGEgcGxvdCBzaG93aW5nIHRoZSBzdW1tYXJ5IG9mIHRoZSBvdmVyYWxsIGNvZWZmaWNpZW50IGluc3RhYmlsaXR5LgoKVG8gZG8gdGhpcyB3ZSB3aWxsIGNhbGN1bGF0ZSB0aGUgc3RhbmRhcmQgZGV2aWF0aW9uIG9mIGNvZWZmaWNpZW50IGVzdGltYXRlcyBmb3IgZWFjaCB2YXJpYWJsZSBhY3Jvc3MgYWxsIG9mIHRoZSBzaW11bGF0aW9ucy4gVGh1cyAgd2Ugd2lsbCBncm91cCAgYnkgIHRoZSBgQW5hbHlzaXNgIGFuZCAgdGhlIGB0ZXJtYCB2YXJpYWJsZXMgbm93IHRoYXQgb3VyIGRhdGEgaXMgaW4gbG9uZyBmb3JtYXQuIFdlIHdpbGwgdXNlIHRoZSBgc2QoKWAgZnVuY3Rpb24gb2YgdGhlIGBzdGF0c2AgcGFja2FnZSB0byBjYWxjdWxhdGUgdGhlIHN0YW5kYXJkIGRldmlhdGlvbi4KCmBgYHtyfQpjb2VmZl9zZCA8LSBzaW11bGF0aW9ucyAlPiUKICBncm91cF9ieShBbmFseXNpcywgdGVybSkgJT4lCiAgc3VtbWFyaXplKCJTRCIgPSBzZChlc3RpbWF0ZSkpCmBgYAoKRmlyc3QsIHdlIHdpbGwgZGlzcGxheSBhbiBpbnRlcmFjdGl2ZSB0YWJsZSBvZiB0aGVzZSBzdGFuZGFyZCBkZXZpYXRpb25zLiBUcnkgc2VhcmNoaW5nIGZvciAiUlRDIiwgYW5kIHlvdSBjYW4gY29tcGFyZSB0aGUgc3RhbmRhcmQgZGV2aWF0aW9ucyBvZiB0aGUgY29lZmZpY2llbnRzIGZvciB0aGUgYFJUQ19MQVdUUlVFYCB2YXJpYWJsZSBhY3Jvc3MgdGhlIHR3byBhbmFseXNlcy4gVG8gdGFrZSBhIGJldHRlciBsb29rIGF0IG91ciBkYXRhIHdlIHdpbGwgdXNlIHRoZSBgZGF0YXRhYmxlKClgIGZ1bmN0aW9uIG9mIHRoZSBgRFRgIHBhY2thZ2Ugd2hpY2ggd2lsbCBjcmVhdGUgYW4gaW50ZXJhY3RpdmUgc2VhcmNoYWJsZSB0YWJsZS4KCmBgYHtyfQpEVDo6ZGF0YXRhYmxlKGNvZWZmX3NkKQpgYGAKCk5vdyB3ZSB3aWxsIG1ha2UgYSBwbG90IG9mIHRoaXMgZGF0YSwgaW5jbHVkaW5nIFNEcyBmcm9tIGFsbCBjb2VmZmljaWVudHMgaW4gZWFjaCBtb2RlbC4KCmBgYHtyfQpzaW11bGF0aW9uX3Bsb3QgPC0gY29lZmZfc2QgJT4lCiAgZ2dwbG90KGFlcyh4ID0gQW5hbHlzaXMsIHkgPSBTRCkpICsKICBnZW9tX2ppdHRlcih3aWR0aCA9IDAuMSwgYWxwaGEgPSAwLjUsIHNpemUgPSAyKSArCiAgbGFicygKICAgIHRpdGxlID0gIkNvZWZmaWNpZW50IHZhcmlhYmlsaXR5IiwKICAgIHN1YnRpdGxlID0gIlNEcyBvZiBjb2VmZmljaWVudCBlc3RpbWF0ZXMgZnJvbSBsZWF2ZS1vbmUtb3V0IGFuYWx5c2lzIiwKICAgIHggPSAiVGVybSIsCiAgICB5ID0gIkNvZWZmaWNpZW50IEVzdGltYXRlIFxuIFN0YW5kYXJkIERldmlhdGlvbnMiLAogICAgY2FwdGlvbiA9ICJSZXN1bHRzIGZyb20gc2ltdWxhdGlvbnMiCiAgKSArCiAgdGhlbWVfbWluaW1hbCgpICsKICB0aGVtZSgKICAgIGF4aXMudGl0bGUueCA9IGVsZW1lbnRfYmxhbmsoKSwKICAgIGF4aXMudGV4dC54ID0gZWxlbWVudF90ZXh0KHNpemUgPSA4LCBjb2xvciA9ICJibGFjayIpLAogICAgYXhpcy50ZXh0LnkgPSBlbGVtZW50X3RleHQoY29sb3IgPSAiYmxhY2siKQogICkKc2ltdWxhdGlvbl9wbG90CmBgYAoKSGVyZSB3ZSBjYW4gY2xlYXJseSBzZWUgdGhhdCBvdmVyYWxsIHRoZSBjb2VmZmljaWVudCBlc3RpbWF0ZXMgYXJlIG11Y2ggbGVzcyBzdGFibGUgaW4gQW5hbHlzaXMgMi4gVGhpcyBpcyBhbiBpbmRpY2F0aW9uIHRoYXQgd2UgbWF5IGhhdmUgbXVsdGljb2xsaW5lYXJpdHkgaW4gb3VyIGRhdGEuCgoKIyMjICoqVklGKioKKioqCgpBbm90aGVyIHdheSBvZiBldmFsdWF0aW5nIHRoZSBwcmVzZW5jZSBhbmQgc2V2ZXJpdHkgb2YgbXVsdGljb2xsaW5lYXJpdHkgaXMgdG8gY2FsY3VsYXRlIHRoZSBbdmFyaWFuY2UgaW5mbGF0aW9uIGZhY3RvciAoVklGKV0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvVmFyaWFuY2VfaW5mbGF0aW9uX2ZhY3Rvcil7dGFyZ2V0PSJfYmxhbmsifSAuIAoKQWNjb3JkaW5nIHRvIFtXaWtpcGVkaWFdKGh0dHBzOi8vZW4ud2lraXBlZGlhLm9yZy93aWtpL1ZhcmlhbmNlX2luZmxhdGlvbl9mYWN0b3Ipe3RhcmdldD0iX2JsYW5rIn06Cgo+SXQgcHJvdmlkZXMgYW4gaW5kZXggdGhhdCBtZWFzdXJlcyBob3cgbXVjaCB0aGUgdmFyaWFuY2UgKHRoZSBzcXVhcmUgb2YgdGhlIGVzdGltYXRlJ3Mgc3RhbmRhcmQgZGV2aWF0aW9uKSBvZiBhbiBlc3RpbWF0ZWQgcmVncmVzc2lvbiBjb2VmZmljaWVudCBpcyBpbmNyZWFzZWQgYmVjYXVzZSBvZiBjb2xsaW5lYXJpdHkuIAoKPiBUaGUgdmFyaWFuY2UgaW5mbGF0aW9uIGZhY3RvciAoVklGKSBpcyB0aGUgKipxdW90aWVudCBvZiB0aGUgdmFyaWFuY2UgaW4gYSBtb2RlbCB3aXRoIG11bHRpcGxlIHRlcm1zIGJ5IHRoZSB2YXJpYW5jZSBvZiBhIG1vZGVsIHdpdGggb25lIHRlcm0gYWxvbmUqKi4KClZJRiB2YWx1ZXMgY2FuIGJlIGNhbGN1bGF0ZWQgZm9yIGVhY2ggZXhwbGFuYXRvcnkgdmFyaWFibGUgaW4gYSBtb2RlbCBieSBwZXJmb3JtaW5nIHRoZSBmb2xsb3dpbmcgY2FsY3VsYXRpb246CgoxKSBSdW4gYW5vdGhlciBvcmRpbmFyeSBsZWFzdCBzcXVhcmVzIChPTFMpIGxpbmVhciByZWdyZXNzaW9uIHdpdGggb25lIG9mIHRoZSBleHBsYW5hdG9yeSB2YXJpYWJsZXMgb2YgeW91ciBtb2RlbCBvZiBpbnRlcmVzdCAoJFhfaSQpIGFzIHRoZSBkZXBlbmRlbnQgdmFyaWFibGUgYW5kIGtlZXAgdGhlIHJlbWFpbmluZyBleHBsYW5hdG9yeSB2YXJpYWJsZXMgYXMgZXhwbGFuYXRvcnkgdmFyaWFibGVzLgoKU28gZ2VuZXJpY2FsbHkgc3BlYWtpbmcgc2F5IHRoaXMgaXMgb3VyIG1vZGVsOgoKJCRZID0gzrJfMCArIM6yXzFYXzEgKyDOsl8yWF8yICsgzrJfM1hfMyArIGUgJCQKCldlIGhhdmUgdGhyZWUgZXhwbGFuYXRvcnkgdmFyaWFibGVzICgkWF8xJCwgJFhfMiQsIGFuZCAkWF8zJCkuCgpJZiB3ZSB3YW50IHRvIGNhbGN1bGF0ZSB0aGUgVklGIHZhbHVlIGZvciAkWF8xJCB3ZSB3b3VsZCBuZWVkIHRvIHBlcmZvcm0gYW5vdGhlciBPTFMgbW9kZWwsIHdoZXJlICRYXzEkIGlzIG5vdyB0aGUgZGVwZW5kZW50IHZhcmlhYmxlIGV4cGxhaW5lZCBieSB0aGUgb3RoZXIgZXhwbGFuYXRvcnkgdmFyaWFibGVzLgoKJCRYXzEgPSDOsl8wICsgIM6yXzJYXzIgKyDOsl8zWF8zICsgZSQkIAoKVGhlIFskUl4yJCBjb2VmZmljaWVudCBvZiBkZXRlcm1pbmF0aW9uXShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9Db2VmZmljaWVudF9vZl9kZXRlcm1pbmF0aW9uKXt0YXJnZXQ9Il9ibGFuayJ9IChhbHNvIGNhbGxlZCBSIHNxdWFyZWQgdmFsdWUpIGZyb20gdGhpcyByZWdyZXNzaW9uIGlzIHRoZW4gdXNlZCB0byBjYWxjdWxhdGUgdGhlIFZJRiBhcyBmb2xsb3dzOgoKJCRcZnJhY3sxfXsxLVJeezJ9fSQkIAoKVGhlICRSXjIkIHZhbHVlIGlzIGluIHRoaXMgY2FzZSB0aGUgcHJvcG9ydGlvbiBvZiB2YXJpYW5jZSBpbiAkWF8xJCBleHBsYWluZWQgYnkgdGhlIG90aGVyIHZhcmlhYmxlcyAoJFhfMiQgYW5kICRYXzMkKS4KCgpWSUYgdmFsdWVzIGFyZSB0eXBpY2FsbHkgY2FsY3VsYXRlZCBmb3IgZWFjaCBleHBsYW5hdG9yeSB2YXJpYWJsZSB3aGVuIGV2YWx1YXRpbmcgbXVsdGljb2xsaW5lYXJpdHkgb2YgYSBtb2RlbC4KClRoZSBjYWxjdWxhdGlvbiBmb3IgYSBzaW5nbGUgdmFyaWFibGUgaXM6CiQkVklGX2kgPSBcZnJhY3sxfXsxLVJfaV57Mn19JCQKV2hlcmUgJGkkIGlzIHRoZSBpbmRleCBvZiBlYWNoIGV4cGxhbmF0b3J5IHZhcmlhYmxlLgoKClJlY2FsbCB0aGF0IGFjY29yZGluZyB0byBbV2lraXBlZGlhXShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9WYXJpYW5jZV9pbmZsYXRpb25fZmFjdG9yKXt0YXJnZXQ9Il9ibGFuayJ9OgoKPiBUaGUgdmFyaWFuY2UgaW5mbGF0aW9uIGZhY3RvciAoVklGKSBpcyB0aGUgKipxdW90aWVudCBvZiB0aGUgdmFyaWFuY2UgaW4gYSBtb2RlbCB3aXRoIG11bHRpcGxlIHRlcm1zIGJ5IHRoZSB2YXJpYW5jZSBvZiBhIG1vZGVsIHdpdGggb25lIHRlcm0gYWxvbmUqKi4KClRoZSAkUl4yJCB2YWx1ZSByYW5nZXMgZnJvbSAwIHRvIDEsIGFuZCBpZiB0aGUgdmFyaWF0aW9uIG9mIG9uZSB2YXJpYWJsZSBpcyBoaWdobHkgZXhwbGFpbmVkIGJ5IHRoZSBvdGhlciB2YXJpYWJsZXMsIHRoZSAkUl4yJCB3aWxsIGFwcHJvYWNoIDEuIFRodXMgdGhlIGRlbm9taW5hdG9yIGluIHRoZSBWSUYgY2FsY3VsYXRpb24gJDEtUl9pXnsyfSQgKHdoaWNoIGlzIHNvbWV0aW1lcyByZWZlcnJlZCB0byBhcyB0b2xlcmFuY2UpIHdpbGwgYmUgc21hbGxlciBhbmQgdGhlIFZJRiB2YWx1ZSB3aWxsIGJlIGxhcmdlci4KClRodXMsICoqaGlnaGVyIFZJRiB2YWxlcyoqIGluZGljYXRlICoqbW9yZSBzZXZlcmUgbXVsdGljb2xsaW5lYXJpdHkqKi4gVHlwaWNhbGx5IGEgdGhyZXNob2xkIG9mIGEgdG9sZXJhbmNlIG9mIGxlc3MgdGhhbiAwLjEwIGFuZC9vciBhIFZJRiBvZiAxMCBvciBhYm92ZSBpcyB1c2VkIGFzIGEgcnVsZSBvZiB0aHVtYiB0byBkZXRlcm1pbmUgaWYgdGhlIHByZXNlbmNlIG9mIG11bHRpY29sbGluZWFyaXR5IG1pZ2h0IGJlIHByb2JsZW1hdGljLgoKUGxlYXNlIHNlZSB0aGlzIFthcnRpY2xlXShodHRwczovL2xpbmsuc3ByaW5nZXIuY29tL2NvbnRlbnQvcGRmLzEwLjEwMDcvczExMTM1LTAwNi05MDE4LTYucGRmKXt0YXJnZXQ9Il9ibGFuayJ9IGZvciBhIHRob3JvdWdoIGV4cGxhbmF0aW9uIG9mIGhvdyB0byBpbnRlcnByZXQgVklGIHZhbHVlcyBhbmQgaG93IHRvIGRlY2lkZSB3aGF0IHRvIGRvIGlmIHlvdXIgbW9kZWwgaGFzIGhpZ2ggbXVsdGljb2xsaW5lYXJpdHkuCgpTbyBob3cgZG8gd2UgY2FsY3VsYXRlIFZJRiB2YWx1ZXMgaW4gUj8KCldlIGNvdWxkIGRvIHRoaXMgbWFudWFsbHkgY3JlYXRpbmcgbWFueSBsaW5lYXIgcmVncmVzc2lvbnMsIGJ1dCB0aGF0IHdvdWxkIG9idmlvdXNseSBiZSB0aW1lIGNvbnN1bWluZy4gTHVja2lseSwgdGhlIGBjYXJgIHBhY2thZ2UgaGFzIGEgZnVuY3Rpb24gY2FsbGVkIGB2aWYoKWAgdGhhdCB3aWxsIGNhbGN1bGF0ZSBWSUYgdmFsdWVzLiBIb3dldmVyLCB0aGVyZSBpcyBvbmUgd3JpbmtsZTogdGhlIGB2aWYoKWAgZnVuY3Rpb24gaXMgbm90IGNvbXBhdGlibGUgd2l0aCB0aGUgb3V0cHV0IG9mIHRoZSBgcGxtYCBmdW5jdGlvbi4gVGhlcmUgaXMgaG93ZXZlciBhIHdvcmthcm91bmQgdGhhdCBhbGxvd3MgdXMgdG8gZml0IGEgc2ltaWxhciBtb2RlbCB1c2luZyB0aGUgc3RhbmRhcmQgYGxtKClgIGZ1bmN0aW9uIG9uIGRhdGEgd2hlcmUgd2UgaGF2ZSByZW1vdmVkIHRoZSB3aXRoaW4taW5kaXZpZHVhbCBtZWFucy4gV2hpbGUgdGhpcyB3b24ndCBnaXZlIHVzIGV4YWN0bHkgdGhlIHNhbWUgcmVzdWx0cyBpbiB0ZXJtcyBvZiB0aGUgc3RhbmRhcmQgZXJyb3JzIG9mIG91ciBlc3RpbWF0ZXMsIGl0IHdpbGwgZ2l2ZSB1cyBzb21lIGlkZWEgb2YgdGhlIFZJRiB2YWx1ZXMgZm9yIHRoZSBjb3ZhcmlhdGVzIGluIG91ciBtb2RlbC4gV2UgYXJlIGZvbGxvd2luZyB0aGUgc3RlcHMgb3V0bGluZWQgW2hlcmVdKGh0dHA6Ly9rYXJ0aHVyLm9yZy8yMDE5L2ltcGxlbWVudGluZy1maXhlZC1lZmZlY3RzLXBhbmVsLW1vZGVscy1pbi1yLmh0bWwpe3RhcmdldD0iX2JsYW5rIn0sIGEgcmVhbGx5IG5pY2Ugc3VtbWFyeSBvZiBwYW5lbCBkYXRhIG1vZGVsaW5nIGluIFIuCgpPbmNlIHdlIGhhdmUgY2FsY3VsYXRlZCBvdXIgVklGIHZhbHVlcywgd2Ugd2lsbCBjcmVhdGUgbmljZXIgbG9va2luZyBvdXRwdXQgb2YgdGhlIGRhdGEgdXNpbmcgdGhlIGBhc190aWJibGUoKWAgZnVuY3Rpb24gb2YgdGhlIGB0aWJibGVgIHBhY2thZ2UgdG8gY3JlYXRlIGEgdGliYmxlIGFuZCBhZGQgdGhlIHZhcmlhYmxlIG5hbWVzIGFzIGFub3RoZXIgY29sdW1uLgoKClJlY2FsbCB0aGF0IHdlIHByZXZpb3VzbHkgY3JlYXRlZCB0aGUgYERPTk9IVUVfT1VUUFVUYCBvYmplY3QgbGlrZSBzbzoKYGBge3J9CkRPTk9IVUVfT1VUUFVUIDwtIHBsbShWaW9sX2NyaW1lX3JhdGVfMWtfbG9nIH4KUlRDX0xBVyArCiAgV2hpdGVfTWFsZV8xNV90b18xOV95ZWFycyArCiAgV2hpdGVfTWFsZV8yMF90b18zOV95ZWFycyArCiAgQmxhY2tfTWFsZV8xNV90b18xOV95ZWFycyArCiAgQmxhY2tfTWFsZV8yMF90b18zOV95ZWFycyArCiAgT3RoZXJfTWFsZV8xNV90b18xOV95ZWFycyArCiAgT3RoZXJfTWFsZV8yMF90b18zOV95ZWFycyArCiAgVW5lbXBsb3ltZW50X3JhdGUgKwogIFBvdmVydHlfcmF0ZSArCiAgUG9wdWxhdGlvbl9sb2cgKwogIHBvbGljZV9wZXJfMTAwa19sYWcsCmVmZmVjdCA9ICJ0d293YXlzIiwKbW9kZWwgPSAid2l0aGluIiwKZGF0YSA9IGRfcGFuZWxfRE9OT0hVRQopCmBgYAoKV2hhdCB3ZSBoYXZlIG1vZGVsZWQgaXMgaG93IGEgc3RhdGUncyB2aW9sZW50IGNyaW1lIHJhdGUgaGFzIGNoYW5nZWQgd2l0aCBtb2RpZmljYXRpb25zIG9mIFJUQyBsYXcgc3RhdHVzIGFuZCBvdmVyIHRpbWUsIHJlbGF0aXZlIHRvIGl0c2VsZiwgYW5kIGhvdyB0aGlzIGNvbXBhcmVzIHRvIHNpbWlsYXIgY2hhbmdlcyBpbiB2aW9sZW50IGNyaW1lIG9mIGFub3RoZXIgc3RhdGUgcmVsYXRpdmUgdG8gaXRzZWxmLiAKClRoZSBjb2VmZmljaWVudHMgZnJvbSB0aGlzIG1vZGVsIHRoZW4gYXJlLCBpbiBhbiBvdmVyc2ltcGxpZmllZCBleHBsYW5hdGlvbiwgY2VudGVyZWQgYWNyb3NzIGFsbCBzdGF0ZXMgYW5kIGFjcm9zcyBhbGwgdGltZSBwb2ludHMuIFRoaXMgaXMgY2FsbGVkICJkZW1lYW5lZCIgZGF0YS4KCldlIHdpbGwgbm93IHVzZSB0aGlzIG1vZGVsIG91dHB1dCB0byBjcmVhdGUgYSBkYXRhIGZyYW1lIG9mIGRlbWVhbmVkIGRhdGEgKHdoZXJlIHRoZSBlZmZlY3Qgb2YgdGltZSBpcyBhY2NvdW50ZWQgZm9yIGFzIGFyZSB0aGUgd2l0aGluLWluZGl2aWR1YWxzIGVmZmVjdHMsIGluIHRoaXMgY2FzZSB0aGUgZGlmZmVyZW50IHN0YXRlcykuICBXZSB3aWxsIG1ha2UgYSBtb2RlbCBtYXRyaXggb2YgdGhpcyBkYXRhIGJ5IHVzaW5nIHRoZSBgbW9kZWwubWF0cml4KClgIGZ1bmN0aW9uIG9mIHRoZSBgc3RhdHNgIHBhY2thZ2UgYW5kIHRoZW4gd2Ugd2lsbCBjcmVhdGUgYSBkYXRhIGZyYW1lIGZyb20gdGhpcyB1c2luZyB0aGUgYmFzZSBgYXMuZGF0YS5mcmFtZSgpYCBmdW5jdGlvbi4KCmBgYHtyfQpsbV9ET05PSFVFX2RhdGEgPC0gYXMuZGF0YS5mcmFtZShtb2RlbC5tYXRyaXgoRE9OT0hVRV9PVVRQVVQpKQpgYGAKCmBgYHtyfQpnbGltcHNlKGxtX0RPTk9IVUVfZGF0YSkKYGBgCgpOb3RpY2UgdGhhdCB0aGlzIGRvZXMgbm90IGNvbnRhaW4gYW55IG91dGNvbWUgZGF0YS4gV2Ugd2lsbCBhZGQgdGhpcyBieSB0YWtpbmcgdGhlIG91dGNvbWUgb2YgdGhlIGBXaXRoaW4oKWAgZnVuY3Rpb24gb2YgdGhlIGBwbG1gIHBhY2thZ2UgdG8gZ2V0IHRoZSB2aW9sZW50IGNyaW1lIGRhdGEgYWZ0ZXIgYWNjb3VudGluZyBmb3IgdGhlIHN0YXRlIHNwZWNpZmljIGVmZmVjdHMuIEFjY29yZGluZyB0byB0aGUgZG9jdW1lbnRhdGlvbiBmb3IgdGhpcyBwYWNrYWdlOgoKPiAgV2l0aGluIHJldHVybnMgYSB2ZWN0b3IgY29udGFpbmluZyB0aGUgdmFsdWVzIGluIGRldmlhdGlvbiBmcm9tIHRoZSBpbmRpdmlkdWFsIG1lYW5zIChpZiBlZmZlY3QgPSAiaW5kaXZpZHVhbCIsIGZyb20gdGltZSBtZWFucyBpZiBlZmZlY3QgPSAidGltZSIpLCB0aGUgc28gY2FsbGVkIGRlbWVhbmVkIGRhdGEuCgpBbHNvIHJlY2FsbCB0aGF0IHRoZSBgZF9wYW5lbF9ET05PSFVFYCBkYXRhIGlzIGp1c3QgdGhlIERvbm9odWUgZGF0YSBpbiBwYW5lbCBmb3JtYXQuIAoKYGBge3J9CmxtX0RPTk9IVUVfZGF0YSAlPD4lCiAgbXV0YXRlKFZpb2xfY3JpbWVfcmF0ZV8xa19sb2cgPSBwbG06OldpdGhpbihwdWxsKAogICAgZF9wYW5lbF9ET05PSFVFLCBWaW9sX2NyaW1lX3JhdGVfMWtfbG9nCiAgKSksIGVmZmVjdCA9ICJ0d293YXlzIikKYGBgCgoKTm93IHdlIHdpbGwgZml0IHRoZSBkZW1lYW5lZCBkYXRhIHRvIHRoZSBtb2RlbDoKYGBge3J9CmxtX0RPTk9IVUUgPC0gbG0oVmlvbF9jcmltZV9yYXRlXzFrX2xvZyB+ClJUQ19MQVdUUlVFICsKICBXaGl0ZV9NYWxlXzE1X3RvXzE5X3llYXJzICsKICBXaGl0ZV9NYWxlXzIwX3RvXzM5X3llYXJzICsKICBCbGFja19NYWxlXzE1X3RvXzE5X3llYXJzICsKICBCbGFja19NYWxlXzIwX3RvXzM5X3llYXJzICsKICBPdGhlcl9NYWxlXzE1X3RvXzE5X3llYXJzICsKICBPdGhlcl9NYWxlXzIwX3RvXzM5X3llYXJzICsKICBVbmVtcGxveW1lbnRfcmF0ZSArCiAgUG92ZXJ0eV9yYXRlICsKICBQb3B1bGF0aW9uX2xvZyArCiAgcG9saWNlX3Blcl8xMDBrX2xhZywKZGF0YSA9IGxtX0RPTk9IVUVfZGF0YQopCmBgYAoKTm93IHdlIGFyZSByZWFkeSB0byB1c2UgdGhlIGB2aWYoKWAgZnVuY3Rpb24gb2YgdGhlIGBjYXJgIHBhY2thZ2UgdG8gY2FsY3VsYXRlIHRoZSBWSUYgdmFsdWVzOgoKYGBge3J9Cgp2aWZfRE9OT0hVRSA8LSB2aWYobG1fRE9OT0hVRSkKCnZpZl9ET05PSFVFCmBgYAoKTm93IHdlIHdpbGwgdXNlIHRoZSBgYXNfdGliYmxlKClgIGZ1bmN0aW9uIG9mIHRoZSBgdGliYmxlYCBwYWNrYWdlIHRvIG5pY2VseSBwdXQgdGhpcyB0b2dldGhlci4gCgpgYGB7cn0KdmlmX0RPTk9IVUUgPC0gdmlmX0RPTk9IVUUgJT4lCiAgYXNfdGliYmxlKCkgJT4lCiAgY2JpbmQoLiwgbmFtZXModmlmX0RPTk9IVUUpKSAlPiUKICBhc190aWJibGUoKQoKY29sbmFtZXModmlmX0RPTk9IVUUpIDwtIGMoIlZJRiIsICJWYXJpYWJsZSIpCgp2aWZfRE9OT0hVRQpgYGAKCk5vdyB3ZSB3aWxsIGRvIHRoZSBzYW1lIGZvciB0aGUgTG90dCBhbmQgTXVzdGFyZCBkYXRhLgoKV2Ugd2lsbCBuZWVkIHRvIHVzZSB0aGUgYHJlbmFtZSgpYCBmdW5jdGlvbiAoeW91IG1heSByZWNhbGwgdGhpcyBpcyBwYXJ0IG9mIHRoZSBgZHBseXJgIHBhY2thZ2UpIHRvIHJlcGxhY2UgYFJUQ19MQVdUUlVFYCB3aXRoIGBSVENfTEFXYCwgYXMgdGhlIHZhcmlhYmxlIG5hbWUgaW4gdGhlIG1vZGVsIG91dHB1dCBpcyBhcHBlbmRlZCBieSBgVFJVRWAgYmVjYXVzZSBpdCB3YXMgYSBsb2dpY2FsIHZhcmlhYmxlLiBCZWNhdXNlIHRoZSBtb2RlbCBmb3JtdWxhIGZvciB0aGUgTG90dCBhbmFseXNpcyBpcyBzbyBjb21wbGV4LCBpdCBpcyBlYXNpZXIgdG8gY2hhbmdlIHRoaXMgdmFyaWFibGUgbmFtZSBpbiBvdXIgbmV3IGRhdGEgZnJhbWUsIHJhdGhlciB0aGFuIHJld3JpdGUgdGhlIGZvcm11bGEgZm9yIHRoaXMgZGF0YS4gCgpSZWNhbGwgdGhhdCB3ZSBhbHJlYWR5IHNhdmVkIHRoZSBmb3JtdWxhIGZvciB0aGlzIGRhdGE6CmBgYHtyfQpMT1RUX2ZtbGEKYGBgCgoKYGBge3J9CmxtX0xPVFRfZGF0YSA8LSBhcy5kYXRhLmZyYW1lKG1vZGVsLm1hdHJpeChMT1RUX09VVFBVVCkpCmxtX0xPVFRfZGF0YSAlPD4lCiAgbXV0YXRlKFZpb2xfY3JpbWVfcmF0ZV8xa19sb2cgPSBwbG06OldpdGhpbihwdWxsKAogICAgZF9wYW5lbF9MT1RULCBWaW9sX2NyaW1lX3JhdGVfMWtfbG9nCiAgKSwgZWZmZWN0ID0gInR3b3dheXMiKSkgJT4lCiAgcmVuYW1lKFJUQ19MQVcgPSBSVENfTEFXVFJVRSkKCmxtX0xPVFQgPC0gbG0oTE9UVF9mbWxhLAogIGRhdGEgPSBsbV9MT1RUX2RhdGEKKQoKdmlmX0xPVFQgPC0gdmlmKGxtX0xPVFQpCgp2aWZfTE9UVAoKdmlmX0xPVFQgPC0gdmlmX0xPVFQgJT4lCiAgYXNfdGliYmxlKCkgJT4lCiAgY2JpbmQoLiwgbmFtZXModmlmX0xPVFQpKSAlPiUKICBhc190aWJibGUoKQoKY29sbmFtZXModmlmX0xPVFQpIDwtIGMoIlZJRiIsICJWYXJpYWJsZSIpCmBgYAoKTm93IHRvIGhhdmUgY29uc2lzdGVudCB2YXJpYWJsZSBuYW1lcyBpbiB0aGUgVklGIGRhdGEgc2V0cyB3ZSB3aWxsIHJlbmFtZSBgUlRDX0xBV2AgYmFjayB0byBgUlRDX0xBV1RSVUVgIHVzaW5nIHRoZSBgc3RyX3JlcGxhY2UoKWAgZnVuY3Rpb24gb2YgdGhlIGBzdHJpbmdyYCBwYWNrYWdlLiBUaGlzIGZ1bmN0aW9uIHJlcGxhY2VzIGEgcGF0dGVybi4KYGBge3J9CnZpZl9MT1RUICU+JSBtdXRhdGUoVmFyaWFibGUgPSBzdHJfcmVwbGFjZSgKICBzdHJpbmcgPSBWYXJpYWJsZSwKICBwYXR0ZXJuID0gIlJUQ19MQVciLAogIHJlcGxhY2VtZW50ID0gIlJUQ19MQVdUUlVFIgopKQp2aWZfTE9UVApgYGAKCgpgYGB7cn0KCkRUOjpkYXRhdGFibGUodmlmX0xPVFQpCmBgYAoKV2UgY2FuIHNlZSB0aGF0IHNvbWUgb2YgdGhlIFZJRiB2YWx1ZXMgYXJlIHZlcnkgaGlnaCEKCk5vdyB3ZSB3aWxsIG1ha2UgYSBwbG90IG9mIHRoZSBWSUYgdmFsdWVzIGZvciBib3RoIGFuYWx5c2VzLiBXZSB3aWxsIGFkZCB0ZXh0IHRvIGEgc3BlY2lmaWMgbG9jYXRpb24gb24gdGhlIHBsb3QgdXNpbmcgdGhlIGBnZW9tX3RleHQoKWAgZnVuY3Rpb24uIFR5cGljYWxseSBhIHRocmVzaG9sZCBvZiAxMCBpcyB1c2VkIHRvIGlkZW50aWZ5IGlmIHRoZSBWSUYgYXJlIHByb2JsZW1hdGljYWxseSBoaWdoLgoKYGBge3J9Cgp2aWZfRE9OT0hVRSAlPD4lCiAgbXV0YXRlKEFuYWx5c2lzID0gIkFuYWx5c2lzIDEiKQp2aWZfTE9UVCAlPD4lCiAgbXV0YXRlKEFuYWx5c2lzID0gIkFuYWx5c2lzIDIiKQoKdmlmX2RmIDwtIGJpbmRfcm93cygKICB2aWZfRE9OT0hVRSwKICB2aWZfTE9UVAopCmRhdGF0YWJsZSh2aWZfZGYpCmBgYAoKWW91IGNhbiBhbHNvIHNlYXJjaCB0aGUgYWJvdmUgdGFibGUgb2YgcmVzdWx0cyBmb3IgIlJUQyIgdG8gc2VlIGhvdyB0aGUgVklGIHZhbHVlcyBkaWZmZXIgZm9yIHRoZSBSVEMgdmFyaWFibGUgYmV0d2VlbiB0aGUgdHdvIGFuYWx5c2VzLiBUaGV5IGFyZSBjbG9zZSB0byBvbmUgYW5vdGhlciwgYWx0aG91Z2ggdGhlIHZhbHVlIGlzIHNsaWdodGx5IGhpZ2hlciBmb3IgQW5hbHlzaXMgMi4KCk5leHQsIHdlIHdpbGwgbWFrZSBhIGNvdXBsZSBvZiBwbG90cyB0byBpbGx1c3RyYXRlIGhvdyB0aGUgVklGIHZhbHVlcyBjb21wYXJlIGJldHdlZW4gdGhlIHR3byBtb2RlbHMuCgpgYGB7cn0KdmlmX3Bsb3QgPC0gdmlmX2RmICU+JQogIGdncGxvdChhZXMoeCA9IEFuYWx5c2lzLCB5ID0gVklGKSkgKwogIGdlb21faml0dGVyKHdpZHRoID0gMC4xLCBhbHBoYSA9IDAuNSwgc2l6ZSA9IDIpICsKICBnZW9tX2hsaW5lKHlpbnRlcmNlcHQgPSAxMCwgY29sb3IgPSAicmVkIikgKwogIGdlb21fdGV4dChhZXMoLjcsIDE4LCBsYWJlbCA9ICJ0eXBpY2FsIGN1dG9mZiBvZiAxMCIpKSArCiAgbGFicyh0aXRsZSA9ICJWYXJpYW5jZSBpbmZsYXRpb24gZmFjdG9ycyIsIHkgPSAiVklGIikgKwogIHRoZW1lX21pbmltYWwoKSArCiAgdGhlbWUoCiAgICBheGlzLnRpdGxlLnggPSBlbGVtZW50X2JsYW5rKCksCiAgICBheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChjb2xvciA9ICJibGFjayIpLAogICAgYXhpcy50ZXh0LnkgPSBlbGVtZW50X3RleHQoY29sb3IgPSAiYmxhY2siKQogICkKCnZpZl9wbG90CmBgYAoKV2UgY2FuIHNlZSB0aGF0IGFuYWx5c2lzIDIgaGFzIHZhcmlhYmxlcyB3aXRoIG11Y2ggaGlnaGVyIG11bHRpY29sbGluZWFyaXR5LgoKCkxldCdzIG1ha2UgdGhpcyBwbG90IGEgbGl0dGxlIGVhc2llciB0byBzZWUgdXNpbmcgdGhlIGBjb29yZF90cmFucygpYCBmdW5jdGlvbiBvZiB0aGUgYGdncGxvdDJgIHBhY2thZ2Ugd2l0aCB0aGUgYHkgPSJsb2cxMCJgIGFyZ3VtZW50LiBUaGlzIGRvZXMgbm90IGNoYW5nZSB0aGUgdmFsdWVzLCBidXQgYWRqdXN0cyB0aGUgd2F5IHRoZSB5IGF4aXMgaXMgZGlzcGxheWVkIHdpdGggZGltaW5pc2hpbmcgZGlzdGFuY2UgYmV0d2VlbiBncmlkIGxpbmVzLiAKCmBgYHtyfQoKdmlmX3Bsb3QgPC0gdmlmX2RmICU+JQogIGdncGxvdChhZXMoeCA9IEFuYWx5c2lzLCB5ID0gVklGKSkgKwogIGdlb21faml0dGVyKHdpZHRoID0gMC4xLCBhbHBoYSA9IDAuNSwgc2l6ZSA9IDIpICsKICBnZW9tX2hsaW5lKHlpbnRlcmNlcHQgPSAxMCwgY29sb3IgPSAicmVkIikgKwogIGdlb21fdGV4dChhZXMoLjc1LCAxMywgbGFiZWwgPSAidHlwaWNhbCBjdXRvZmYgb2YgMTAiKSkgKwogIGNvb3JkX3RyYW5zKHkgPSAibG9nMTAiKSArCiAgbGFicyh0aXRsZSA9ICJWYXJpYW5jZSBpbmZsYXRpb24gZmFjdG9ycyIpICsKICB0aGVtZV9taW5pbWFsKCkgKwogIHRoZW1lKAogICAgYXhpcy50aXRsZS54ID0gZWxlbWVudF9ibGFuaygpLAogICAgYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoY29sb3IgPSAiYmxhY2siKSwKICAgIGF4aXMudGV4dC55ID0gZWxlbWVudF90ZXh0KGNvbG9yID0gImJsYWNrIikKICApCgp2aWZfcGxvdApgYGAKCgpJbiBtYW55IGNhc2VzIGl0IHdvdWxkIGJlIGFkdmlzYWJsZSB0byByZW1vdmUgb25lIG9yIG1vcmUgb2YgdGhlc2UgdmFyaWFibGVzIGFuZCByZWFzc2VzcyB0aGUgVklGIHZhbHVlcy4gVGhlcmUgYXJlIGFsc28gb3RoZXIgb3B0aW9ucywgc3VjaCBhcyBbcmlkZ2UgcmVncmVzc2lvbl0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvVGlraG9ub3ZfcmVndWxhcml6YXRpb24pe3RhcmdldD0iX2JsYW5rIn0uIEhvd2V2ZXIsIGJvdGggb2YgdGhlc2Ugb3B0aW9ucyBuZWVkIHRvIGJlIGRvbmUgd2l0aCBjYXJlIGFzIHRoZXkgY2FuIGFsc28gaW50cm9kdWNlIGJpYXMgaW50byB0aGUgbW9kZWwuCgpJbiBhbnkgY2FzZSB0aGUgcHJlc2VuY2Ugb2YgbXVsdGljb2xsaW5lYXJpdHkgc2hvdWxkIGVuY291cmFnZSBmdXJ0aGVyIGludmVzdGlnYXRpb24gYWJvdXQgdGhlIGRlc2lnbiBvZiB0aGUgbW9kZWwsIGFzIHRoZSByZXN1bHRzIG1heSBub3QgYmUgcmVsaWFibGUgZHVlIHRvIHRoZSBpbmNyZWFzZWQgbGV2ZWwgb2YgaW5zdGFiaWxpdHkgb2YgdGhlIGNvZWZmaWNpZW50IGVzdGltYXRlcy4KClNlZSB0aGlzIFthcnRpY2xlXShodHRwczovL2xpbmsuc3ByaW5nZXIuY29tL2NvbnRlbnQvcGRmLzEwLjEwMDcvczExMTM1LTAwNi05MDE4LTYucGRmKXt0YXJnZXQ9Il9ibGFuayJ9IGZvciBhIGRldGFpbGVkIGRpc2N1c3Npb24gYWJvdXQgd2hhdCB0byBjb25zaWRlciB3aGVuIHlvdXIgbW9kZWwgaGFzIHZhcmlhYmxlcyB3aXRoIGhpZ2ggVklGIHZhbHVlcy4gCgoKIyAqKkRhdGEgVmlzdWFsaXphdGlvbioqCioqKiAKCk5vdyBsZXRzIG1ha2UgYSBwbG90IHRoYXQgc3VtbWFyaXplcyBhbGwgb2Ygb3VyIGZpbmRpbmdzLgoKV2Ugd2lsbCB1c2UgdGhlIGBjb3dwbG90YCBwYWNrYWdlIHRvIHB1dCBvdXIgcGxvdHMgdG9nZXRoZXIuIAoKV2Ugd2lsbCB1c2UgdGhlIGBnZ2RyYXcoKWAgZnVuY3Rpb24gb2YgdGhpcyBwYWNrYWdlLiBUaGlzIGFsbG93cyB5b3UgdG8gYWRkIGxhYmVscyBhbmQgb3RoZXIgcGxvdCBhc3BlY3RzIG9uIHRvcCBvZiBleGlzdGluZyBwbG90cy4gVGh1cyBpZiB3ZSB3YW50IHRvIGFkZCBhIHRpdGxlIGVsZW1lbnQgdG8gb3VyIG92ZXJhbGwgcGxvdCB0aGF0IHdlIHdpbGwgYWRkIHRvIG91ciBjb21iaW5lZCBwbG90IHdlIGNhbiB1c2UgYGdnZHJhdygpYCB0byBzdGFydCBhbmQgdGhlbiB0aGUgYGRyYXdfbGFiZWwoKWAgZnVuY3Rpb24gdG8gYWRkIHRleHQuCgoKYGBge3IsIGZpZy5oZWlnaHQ9MTAsIGVjaG89RkFMU0UsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0V9CnRpdGxlX3Bsb3RzIDwtIGdnZHJhdygpICsKICBkcmF3X2xhYmVsKAogICAgIk11bHRpY29sbGluZWFyaXR5IGFuZCBpdHMgZWZmZWN0cyIsCiAgICBmb250ZmFjZSA9ICJib2xkIiwKICAgIHNpemUgPSAxOCwKICAgIHggPSAwLAogICAgaGp1c3QgPSAtMC4wMQogICkgKwogIHRoZW1lKAogICAgcGxvdC5tYXJnaW4gPSBtYXJnaW4oMCwgMCwgMCwgMCkKICApCgp0aXRsZV9wbG90cwpjbGFzcyh0aXRsZV9wbG90cykKYGBgCgpBcyB5b3UgY2FuIHNlZSB3ZSBrbm93IGhhdmUgcGxvdCBvYmplY3QgdGhhdCBqdXN0IGhhcyB0aGUgdGV4dCBgIk11bHRpY29sbGluZWFyaXR5IGFuZCBpdHMgZWZmZWN0cyJgLgoKTm93IHdlIHdpbGwgY3JlYXRlIGEgc3VidGl0bGUgaW4gdGhlIHNhbWUgd2F5LgoKYGBge3J9Cgpmb3J3YXJkIDwtIGdnZHJhdygpICsKICBkcmF3X2xhYmVsKAogICAgIkFuYWx5c2lzIDE6IDYgZGVtb2dyYXBoaWMgdmFyaWFibGVzXG5BbmFseXNpcyAyOiAzNiBkZW1vZ3JhcGhpYyB2YXJpYWJsZXMiLAogICAgZm9udGZhY2UgPSAiYm9sZCIsCiAgICBzaXplID0gMTAsCiAgICB4ID0gMCwKICAgIGhqdXN0ID0gLTAuMDIKICApICsKICB0aGVtZSgKICAgIHBsb3QubWFyZ2luID0gbWFyZ2luKDAsIDAsIDAsIDApCiAgKQoKZm9yd2FyZApgYGAKCk5vdyB3ZSB3aWxsIHJlY3JlYXRlIG91ciBjb3JyZWxhdGlvbiBwbG90cyB3aXRoIHNvbWUgc2xpZ2h0IGFsdGVyYXRpb25zLiBXZSB3YW50IHRvIHJlbW92ZSBvdXIgbGFiZWxzLCBiZWNhdXNlIHRoZXkgd2lsbCBiZSB0b28gc21hbGwgdG8gc2VlIHdoZW4gd2UgY29tYmluZSBvdXIgcGxvdHMuIFRvIGRvIHRoaXMgd2Ugd2lsbCB1c2UgdGhlIGB0aGVtZV92b2lkKClgIGZ1bmN0aW9uIG9mIHRoZSBgZ2dwbG90MmAgcGFja2FnZS4KCk5vdGUgdGhhdCBiZWNhdXNlIHdlIGFyZSBsYXllcmluZyBnZ3Bsb3QyIG9iamVjdHMgd2UgY2FuJ3QgdXNlIHRoZSBgJT4lYCBwaXBlIHRvIHN0YXJ0IHdpdGggdGhlIGV4aXN0aW5nIGNvcnJlbGF0aW9uIHBsb3RzLgoKYGBge3J9Cgpjb3JyX21hdF9ET05PSFVFIDwtIGNvcnJfbWF0X0RPTk9IVUUgKwogIHRoZW1lX3ZvaWQoKSArCiAgdGhlbWUocGxvdC50aXRsZSA9IGVsZW1lbnRfdGV4dChzaXplID0gOCwgY29sb3IgPSAiYmxhY2siKSkgKwogIGxhYnModGl0bGUgPSAiQW5hbHlzaXMgMSIpCgpjb3JyX21hdF9MT1RUIDwtIGNvcnJfbWF0X0xPVFQgKwogIHRoZW1lX3ZvaWQoKSArCiAgdGhlbWUocGxvdC50aXRsZSA9IGVsZW1lbnRfdGV4dChzaXplID0gOCwgY29sb3IgPSAiYmxhY2siKSkgKwogIGxhYnModGl0bGUgPSAiQW5hbHlzaXMgMiIpCmBgYAoKT0sgd2Ugd2FudCB0byBhcnJhbmdlIG91ciBjb3JyZWxhdGlvbiBwbG90cyB0byBiZSBpbiB0aGUgdG9wIHJvdyBvZiBvdXIgbGFyZ2VyIHBsb3QuIE5vdyB3ZSB3aWxsIHVzZSB0aGUgYHBsb3RfZ3JpZCgpYCBmdW5jdGlvbiB0byBhcnJhbmdlIHRoZSBwbG90cy4KCmBgYHtyfQpyb3dfQSA8LSBwbG90X2dyaWQoY29ycl9tYXRfRE9OT0hVRSwKICBjb3JyX21hdF9MT1RULAogIG5yb3cgPSAxCikKcm93X0EKYGBgCgoKTmljZSEgV2UgaGF2ZSBjb21iaW5lZCBwbG90cyEKCgpOb3cgbGV0J3MgYWRkIGEgdGl0bGUgZm9yIHRoZXNlIHBsb3RzLgoKYGBge3J9Cgp0aXRsZV9BIDwtIGdnZHJhdygpICsKICBkcmF3X2xhYmVsKAogICAgIkNvcnJlbGF0aW9uIGJldHdlZW4gdmFyaWFibGVzIGNhbiBpbmR1Y2UgbXVsdGljb2xsaW5lYXJpdHkiLAogICAgZm9udGZhY2UgPSAiYm9sZCIsCiAgICBzaXplID0gMTQsCiAgICB4ID0gMCwKICAgIGhqdXN0ID0gLTAuMDEKICApICsKICB0aGVtZSgKICAgIHBsb3QubWFyZ2luID0gbWFyZ2luKDAsIDAsIDAsIDApCiAgKQoKcGxvdF9BIDwtIHBsb3RfZ3JpZCh0aXRsZV9BLAogIHJvd19BLAogIG5jb2wgPSAxLAogIHJlbF9oZWlnaHRzID0gYygwLjEsIDEpCikKYGBgCgoKRm9yIG91ciBzZWNvbmQgcm93IGluIG91ciBsYXJnZXIgcGxvdCB3ZSB3YW50IHRvIGhhdmUgdGhlIGZvcm11bGEgZm9yIGNhbGN1bGF0aW5nIFZJRiB2YWx1ZXMgb24gdGhlIGxlZnQgYW5kIHRoZSBwbG90IHRoYXQgd2UgY3JlYXRlZCBwcmV2aW91c2x5IHNob3dpbmcgVklGIHZhbHVlcyBvbiB0aGUgcmlnaHQuCgpGaXJzdCB3ZSB3aWxsIGNyZWF0ZSBhIHBsb3Qgb2JqZWN0IHRoYXQganVzdCBoYXMgdGhlIGZvcm11bGEuCgpUbyBkbyBzbyB3ZSBhcmUgZ29pbmcgdG8gY3JlYXRlIGEgcGxvdCB3aXRoIGEgbGFyZ2UgbGFiZWwgaW4gdGhlIG1pZGRsZSBjb250YWluaW5nIHRoZSBmb3JtdWxhLiBUaGVuIHdlIHdpbGwgdXNlIHRoZSBgdGhlbWVfdm9pZCgpYCBmdW5jdGlvbiBhZ2FpbiB0byByZW1vdmUgdGhlIGF4aXMgbGFiZWxzIGFuZCBiYWNrZ3JvdW5kLgoKVG8gY3JlYXRlIG91ciBwbG90IHdlIHdpbGwgZmlyc3QgcGxvdCB2YWx1ZXMgZnJvbSAxLTEwIGZvciBib3RoIHRoZSB4IGFuZCB5IGF4aXMsIGFsbG93aW5nIHVzIHRvIGNlbnRlciB0aGUgZm9ybXVsYSBhdCB0aGUgeCBhbmQgeSB2YWx1ZXMgb2YgNS4KClRvIHR5cGUgdGhlIGZvcm11bGEgd2Ugd2lsbCB1c2UgW0xhVGVYIG1hdGhlbWF0aWNhbCBub3RhdGlvbl0oaHR0cHM6Ly93d3cuY2FsdmluLmVkdS9+cnBydWltL2NvdXJzZXMvczM0MS9TMTcvZnJvbS1jbGFzcy9NYXRoaW5SbWQuaHRtbCl7dGFyZ2V0PSJfYmxhbmsifS4KClRoZSBzdGFydCBhbmQgZW5kIG9mIGlubGluZSBtYXRoZW1hdGljYWwgZm9ybXVsYXMgYXJlIHNwZWNpZmllZCB1c2luZyBkb2xsYXIgc2lnbnMgKGAkYCkuICAgClN1YnNjcmlwdHMgYXJlIHdyaXR0ZW4gYnkgdXNpbmcgYW4gdW5kZXJzY29yZSAoYF9gKSBhbmQgYnJhY2tldHMgKGB7fWApIGluZGljYXRlIHRoZSBzdGFydCBhbmQgZW5kIG9mIHRoZSBzdWJzY3JpcHQuICAgCgpGcmFjdGlvbnMgYXJlIGluZGljYXRlZCB1c2luZyBgXGZyYWN7bnVtZXJhdG9yfXtkZW5vbWluYXRvcn1gLgoKU3VwZXJzY3JpcHRzIGFyZSBjcmVhdGVkIHVzaW5nIHRoZSBjYXJyb3Qgc3ltYm9sIChgXmApIGFuZCBicmFja2V0cyAoYHt9YCkgaW5kaWNhdGUgdGhlIHN0YXJ0IGFuZCBlbmQgb2YgdGhlIHN1cGVyc2NyaXB0LiAKCkdyZWVrIGxldHRlcnMgIGFyZSBjcmVhdGVkIGJ5IHVzaW5nLCBmb3IgZXhhbXBsZSwgYFxiZXRhYC4KCkluIHRoZSBjYXNlIG9mIHRoZSBmcmFjdGlvbiBhbmQgR3JlZWsgbGV0dGVycyBhbiBhZGRpdGlvbmFsIGBcYCBpcyBuZWVkZWQgaW4gdGhlIGBUZXgoKWAgZnVuY3Rpb24uCgpXZSB3aWxsIHVzZSB0aGUgYFRlWCgpYCBmdW5jdGlvbiBvZiB0aGUgYGxhdGV4MmV4cGAgcGFja2FnZSB0byBjb252ZXJ0IG91ciBMYVRlWCBzdHJpbmcgdG8gYSBbcGxvdG1hdGggZXhwcmVzc2lvbl0oaHR0cHM6Ly9zdGF0LmV0aHouY2gvUi1tYW51YWwvUi1kZXZlbC9saWJyYXJ5L2dyRGV2aWNlcy9odG1sL3Bsb3RtYXRoLmh0bWwpe3RhcmdldD0iX2JsYW5rIn0gKGEgbWF0aGVtYXRpY2FsIG5vdGF0aW9uIGluIFIgdG8gYmUgdXNlZCBpbiBwbG90cykuCgpgYGB7cn0KZW1wdHlfZGYgPC0gY2JpbmQoYygxOjEwKSwgYygxOjEwKSkgJT4lCiAgYXMuZGF0YS5mcmFtZSgpCgpjb2xuYW1lcyhlbXB0eV9kZikgPC0gYygiWCIsICJZIikKCnBsb3RfQjEgPC0gZ2dwbG90KGVtcHR5X2RmLCBhZXMoeCA9IFgsIHkgPSBZKSkgKwogIGFubm90YXRlKCJ0ZXh0IiwKICAgIHggPSA1LAogICAgeSA9IDgsCiAgICBsYWJlbCA9IFRlWCgiJFhfezF9ID0gXFxiZXRhX3swfSArIFxcYmV0YV97Mn1YX3syfSArIFxcYmV0YV97M31YX3szfS4uLitcXGJldGFfe2t9WF97a30rZSQiKSwKICAgIHNpemUgPSA3CiAgKSArCiAgeWxpbSgwLCAxMCkgKwogIHhsaW0oMCwgMTApICsKICBhbm5vdGF0ZSgidGV4dCIsCiAgICB4ID0gNS45LAogICAgeSA9IDUuNSwKICAgIGxhYmVsID0gVGVYKCIkUl8xXnsyfSQiKSwKICAgIHNpemUgPSA3CiAgKSArCiAgeWxpbSgwLCAxMCkgKwogIHhsaW0oMCwgMTApICsKICBnZW9tX3NlZ21lbnQoYWVzKHggPSA1LCB5ID0gNiwgeGVuZCA9IDUsIHllbmQgPSA0LjUpLAogICAgYXJyb3cgPSBhcnJvdyhhbmdsZSA9IDQ1LCBlbmRzID0gImxhc3QiLCB0eXBlID0gIm9wZW4iKSwKICAgIHNpemUgPSAxLjgsCiAgICBjb2xvciA9ICJibGFjayIsCiAgICBsaW5lZW5kID0gImJ1dHQiLAogICAgbGluZWpvaW4gPSAibWl0cmUiCiAgKSArCiAgYW5ub3RhdGUoInRleHQiLAogICAgeCA9IDUsCiAgICB5ID0gMiwKICAgIGxhYmVsID0gVGVYKCIkVklGX3sxfSA9IFxcZnJhY3sxfXsxLVJfezF9XnsyfX0kIiksCiAgICBzaXplID0gNwogICkKCgoKCnBsb3RfQjEKCnBsb3RfQjEgPC0gcGxvdF9CMSArCiAgdGhlbWVfdm9pZCgpCgpwbG90X0IxCmBgYAoKTm93IHdlIHdpbGwgY29tYmluZSB0aGlzIHdpdGggdGhlIFZJRiBwbG90LgoKYGBge3J9CnBsb3RfQjIgPC0gdmlmX3Bsb3QgKwogIHRoZW1lKGF4aXMudGV4dC54ID0gZWxlbWVudF90ZXh0KHNpemUgPSA4KSkKCnJvd19CIDwtIHBsb3RfZ3JpZChwbG90X0IxLAogIHBsb3RfQjIsCiAgbnJvdyA9IDEKKQoKdGl0bGVfQiA8LSBnZ2RyYXcoKSArCiAgZHJhd19sYWJlbCgKICAgICJWYXJpYW5jZSBpbmZsYXRpb24gZmFjdG9ycyBjYW4gYmUgdXNlZCB0byBpZGVudGlmeSBtdWx0aWNvbGxpbmVhcml0eSB3aGVuIHByZXNlbnQiLAogICAgZm9udGZhY2UgPSAiYm9sZCIsCiAgICBzaXplID0gMTQsCiAgICB4ID0gMCwKICAgIGhqdXN0ID0gLS4wMSwKICApICsKICB0aGVtZSgKICAgIHBsb3QubWFyZ2luID0gbWFyZ2luKDAsIDAsIDAsIDApCiAgKQoKcGxvdF9CIDwtIHBsb3RfZ3JpZCh0aXRsZV9CLAogIHJvd19CLAogIG5jb2wgPSAxLAogIHJlbF9oZWlnaHRzID0gYygwLjEsIDEpCikKCnBsb3RfQgpgYGAKCgpOb3cgZm9yIHRoZSB0aGlyZCByb3cgd2Ugd2FudCB0byBpbmNsdWRlIHRoZSBgY29tcGFyaW5nX2FuYWx5c2VzX3Bsb3RgIGFuZCB0aGUgYHNpbXVsYXRpb25fcGxvdGAuCgpgYGB7cn0KcGxvdF9DMSA8LSBjb21wYXJpbmdfYW5hbHlzZXNfcGxvdCArCiAgdGhlbWUoCiAgICBheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChzaXplID0gOCksCiAgICBheGlzLnRpdGxlLnggPSBlbGVtZW50X2JsYW5rKCkKICApICsKICBsYWJzKAogICAgdGl0bGUgPSAiUmVzdWx0cyBpbiBkaWZmZXJlbnQgZXN0aW1hdGVzIiwKICAgIHN1YnRpdGxlID0gIkRpZmZlcmVudCBkZW1vZ3JhcGhpYyBncm91cGluZ3MgY2FuIGNoYW5nZSBkaXJlY3Rpb24gb2YgZXN0aW1hdGUiCiAgKQoKcGxvdF9DMiA8LSBzaW11bGF0aW9uX3Bsb3QgKwogIGxhYnModGl0bGUgPSAiUmVkdWNlcyBwcmVjaXNpb24gaW4gZXN0aW1hdGVzIikKCnJvd19DIDwtIHBsb3RfZ3JpZChwbG90X0MxLAogIHBsb3RfQzIsCiAgbnJvdyA9IDEKKQoKdGl0bGVfQyA8LSBnZ2RyYXcoKSArCiAgZHJhd19sYWJlbCgKICAgICJNdWx0aWNvbGxpbmVhcml0eSBjYW4gaGF2ZSBhbiBlZmZlY3Qgb24gc3RhdGlzdGljYWwgaW5mZXJlbmNlIiwKICAgIGZvbnRmYWNlID0gImJvbGQiLAogICAgc2l6ZSA9IDE0LAogICAgeCA9IDAsCiAgICBoanVzdCA9IC0wLjAxCiAgKSArCiAgdGhlbWUoCiAgICBwbG90Lm1hcmdpbiA9IG1hcmdpbigwLCAwLCAwLCAwKQogICkKCnBsb3RfQyA8LSBwbG90X2dyaWQodGl0bGVfQywKICByb3dfQywKICBuY29sID0gMSwKICByZWxfaGVpZ2h0cyA9IGMoMC4xLCAxKQopCgpwbG90X0MKYGBgCgoKTm93IHRoYXQgd2UgaGF2ZSBhbGwgb2Ygb3VyIHJvd3Mgd2UgY2FuIGNvbWJpbmUgZXZlcnl0aGluZyB0b2dldGhlci4gV2Ugd2lsbCBhbHNvIG1ha2UgdGhlIGJhY2tncm91bmQgb2YgdGhlIHBsb3Qgd2hpdGUsIHdoaWNoIHdvdWxkIGJlIHRyYW5zcGFyZW50IG90aGVyd2lzZS4KCmBgYHtyfQpwbG90cyA8LSBwbG90X2dyaWQocGxvdF9BLAogIHBsb3RfQiwKICBwbG90X0MsCiAgbmNvbCA9IDEsCiAgcmVsX2hlaWdodHMgPSBjKDEsIDEsIDEpKSAKCgptYWlucGxvdCA8LSBwbG90X2dyaWQodGl0bGVfcGxvdHMsCiAgZm9yd2FyZCwKICBwbG90cywKICBuY29sID0gMSwKICByZWxfaGVpZ2h0cyA9IGMoCiAgICAwLjA1LAogICAgMC4wNSwKICAgIDEKICApCikgKwogIHRoZW1lKHBsb3QuYmFja2dyb3VuZCA9IGVsZW1lbnRfcmVjdChmaWxsID0gIndoaXRlIikpCgptYWlucGxvdApgYGAKCgoKCmBgYHtyLCBlY2hvPUZBTFNFLCBpbmNsdWRlPUZBTFNFfQpnZ3NhdmUoaGVyZTo6aGVyZSgiaW1nIiwgIm1haW5wbG90LnBuZyIpKQpgYGAKCgoKCiMgKipTdW1tYXJ5KioKKioqIAoKVGhpcyBjYXNlIHN0dWR5IGhhcyBpbnRyb2R1Y2VkIHRoZSBjb25jZXB0IG9mIG11bHRpY29sbGluZWFyaXR5IGJ5IGV4cGxvcmluZyBkYXRhIHJlbGF0ZWQgdG8gdmlvbGVudCBjcmltZXMgYW5kIHJpZ2h0LXRvLWNhcnJ5IGd1biBsYXdzLiBXZSBhbHNvIGludHJvZHVjZWQgdGhlIHRvcGljIG9mIHBhbmVsIGRhdGEgYXMgYSBzcGVjaWFsIHR5cGUgb2YgbG9uZ2l0dWRpbmFsIGRhdGEgdGhhdCBpbmNsdWRlcyBkYXRhIG9mIDIgb3IgbW9yZSBpbmRpdmlkdWFscyBvciBncm91cHMgb3ZlciAyIG9yIG1vcmUgdGltZSBwb2ludHMuIFdlIGxlYXJuZWQgdGhhdCB3ZSBjYW4gdXNlIHRoZSBgcGxtYCBwYWNrYWdlIHRvIHBlcmZvcm0gcGFuZWwgbGluZWFyIHJlZ3Jlc3Npb24gYW5hbHlzaXMuIFdlIGxlYXJuZWQgdGhhdCB0aGUgZml4ZWQgZWZmZWN0IG1vZGVsIGluIHBhbmVsIGFuYWx5c2lzIGFjdHVhbGx5IG1ha2VzIHRoZSBsZWFzdCBhc3N1bXB0aW9ucywgYW5kIGlzIHRoZXJlZm9yZSBvZnRlbiB0aGUgbW9zdCBhcHByb3ByaWF0ZSB0ZXN0LgoKQnkgZXZhbHVhdGluZyB0d28gYW5hbHlzZXMgdGhhdCB3ZXJlIGlkZW50aWNhbCBleGNlcHQgZm9yIHRoZSBpbmNsdXNpb24gb2YgZXh0cmEgZGVtb2dyYXBoaWMgdmFyaWFibGVzIChBbmFseXNpcyAxIGluY2x1ZGVkIDYsIHdoaWxlIEFuYWx5c2lzIDIgaW5jbHVkZWQgMzYpLCB3ZSBkaXNjb3ZlcmVkIHRoYXQgcmVkdW5kYW50IGFuZCBjb2xsaW5lYXIgdmFyaWFibGVzIGNhbiBjaGFuZ2UgdGhlIGRpcmVjdGlvbmFsaXR5IGFuZCBtYWduaXR1ZGUgb2Ygb3VyIGZpbmRpbmdzLiAKCldlIGxlYXJuZWQgdGhhdCBieSBsb29raW5nIGF0IHRoZSBjb3JyZWxhdGlvbiBiZXR3ZWVuIHBhaXJzIG9mIGV4cGxhbmF0b3J5IHZhcmlhYmxlcyB3ZSBjYW4gZ2V0IGEgc2Vuc2UgYWJvdXQgd2hldGhlciBtdWx0aWNvbGxpbmVhcml0eSBtYXkgZXhpc3QgaW4gb3VyIGRhdGEuCgpXZSBsZWFybmVkIHRoYXQgd2UgY2FuIGV2YWx1YXRlIHRoZSBzdGFiaWxpdHkgb2Ygb3VyIGNvZWZmaWNpZW50IGVzdGltYXRlcyBhY3Jvc3Mgc3ViLXNhbXBsZXMgb3IgY2FsY3VsYXRlIHZhcmlhbmNlIGluZmxhdGlvbiBmYWN0b3IgKFZJRikgdmFsdWVzIHRvIGdldCBhIHNlbnNlIG9mIHRoZSBwcmVzZW5jZSBhbmQgc2V2ZXJpdHkgb2YgbXVsdGljb2xsaW5lYXJpdHkuCgpXZSBsZWFybmVkIHRoYXQgb2Z0ZW4gYSBydWxlIG9mIHRodW1iIG9mID4xMCBpcyB1c2VkIGFzIGEgdGhyZXNob2xkIGZvciByYWlzaW5nIGNvbmNlcm4gYWJvdXQgdGhlIHNldmVyaXR5IG9mIG11bHRpY29sbGluZWFyaXR5LiBIb3dldmVyLCB3ZSBhbHNvIGxlYXJuZWQgdGhhdCAoYXMgb2Z0ZW4gaXMgdGhlIGNhc2Ugd2l0aCB0aHJlc2hvbGRzKSBtb3JlIGNhcmUgbWF5IGJlIHJlcXVpcmVkLiAKCk92ZXJhbGwgd2UgbGVhcm5lZCB0aGF0IG11bHRpY29sbGluZWFyaXR5IGNhbiBiaWFzIG91ciByZWdyZXNzaW9uIGZpbmRpbmdzIGFuZCBpdCBpcyBnb29kIHByYWN0aWNlIHRvIGNoZWNrIGZvciBtdWx0aWNvbGxpbmVhcml0eSB3aGVuIHBlcmZvcm1pbmcgcmVncmVzc2lvbiBhbmFseXNpcy4gIEl0IGlzIHNvbWV0aGluZyB0byBrZWVwIGluIG1pbmQgd2hlbiB3ZSBlbmNvdW50ZXIgY29lZmZpY2llbnQgZXN0aW1hdGVzIHRoYXQgYXJlIHVuZXhwZWN0ZWQuIAoKSW1wb3J0YW50bHkgdGhpcyBjYXNlIHN0dWR5IHNob3djYXNlcyBob3cgbWV0aG9kb2xvZ2ljYWwgZGV0YWlscywgbGlrZSBob3cgd2UgZGVjaWRlIHRvIHBhcnNlIG91ciBkZW1vZ3JhcGhpYyB2YXJpYWJsZXMsIGNhbiBoYXZlIGdyZWF0IGNvbnNlcXVlbmNlcyBvbiB0aGUgcmVzdWx0cyBvZiBvdXIgYW5hbHlzZXMuIAoKCiMgKipTdWdnZXN0ZWQgSG9tZXdvcmsqKgoqKiogCgpBc2sgc3R1ZGVudHMgdG8gcmVtb3ZlIG9uZSBvciBtb3JlIG9mIHRoZSBkZW1vZ3JhcGhpYyB2YXJpYWJsZXMgd2l0aCBoaWdoIFZJRiB2YWx1ZXMgZnJvbSB0aGUgTXVzdGFyZCBhbmQgTG90dC1saWtlIHBhbmVsIGRhdGEgYW5kIHBlcmZvcm0gdGhlIHBhbmVsIGxpbmVhciByZWdyZXNzaW9uIGFuYWx5c2lzIGFnYWluLCBhcyB3ZWxsIGFzIGNhbGN1bGF0ZSB0aGUgVklGIHZhbHVlcy4gCgpBc2sgdGhlIHN0dWRlbnRzIHRvIGRpc2N1c3MgaG93IHRoaXMgcG9zc2libHkgY2hhbmdlZCB0aGUgcmVzdWx0cy4KCgojICoqQWRkaXRpb25hbCBJbmZvcm1hdGlvbioqCioqKgoKIyMgKipIZWxwZnVsIExpbmtzKioKKioqCgpbVGlkeXZlcnNlXShodHRwczovL3d3dy50aWR5dmVyc2Uub3JnLyl7dGFyZ2V0PSJfYmxhbmsifSAgClBsZWFzZSBzZWUgW3RoaXMgY2FzZSBzdHVkeV0oaHR0cHM6Ly9vcGVuY2FzZXN0dWRpZXMuZ2l0aHViLmlvL29jcy1icC1jbzItZW1pc3Npb25zLyl7dGFyZ2V0PSJfYmxhbmsifSAgZm9yIG1vcmUgZGV0YWlscyBvbiB1c2luZyBgZ2dwbG90MmAgICAgIApbTG9uZ2l0dWRpbmFsIHN0dWRpZXNdKGh0dHBzOi8vd3d3LmJtai5jb20vYWJvdXQtYm1qL3Jlc291cmNlcy1yZWFkZXJzL3B1YmxpY2F0aW9ucy9lcGlkZW1pb2xvZ3ktdW5pbml0aWF0ZWQvNy1sb25naXR1ZGluYWwtc3R1ZGllcyl7dGFyZ2V0PSJfYmxhbmsifSAgIApbUGFuZWwgZGF0YV0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvUGFuZWxfZGF0YSl7dGFyZ2V0PSJfYmxhbmsifSAgICAKW0NvbmZpZGVuY2UgaW50ZXJ2YWxzXShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9Db25maWRlbmNlX2ludGVydmFsKXt0YXJnZXQ9Il9ibGFuayJ9ICAgCltMaW5lYXIgcmVncmVzc2lvbl0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvTGluZWFyX3JlZ3Jlc3Npb24pe3RhcmdldD0iX2JsYW5rIn0gIApbcGFuZWwgcmVncmVzc2lvbiBhbmFseXNpc10oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvUGFuZWxfYW5hbHlzaXMpe3RhcmdldD0iX2JsYW5rIn0gICAKW0hhdXNtYW4gdGVzdF0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvRHVyYmluJUUyJTgwJTkzV3UlRTIlODAlOTNIYXVzbWFuX3Rlc3Qpe3RhcmdldD0iX2JsYW5rIn0gCltSZXNhbXBsaW5nXShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9SZXNhbXBsaW5nXyhzdGF0aXN0aWNzKSl7dGFyZ2V0PSJfYmxhbmsifSAgIApbVmFyaWFuY2UgaW5mbGF0aW9uIGZhY3RvciAoVklGKV0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvVmFyaWFuY2VfaW5mbGF0aW9uX2ZhY3Rvcil7dGFyZ2V0PSJfYmxhbmsifSAgIApbJFJeMiQgY29lZmZpY2llbnQgb2YgZGV0ZXJtaW5hdGlvbl0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvQ29lZmZpY2llbnRfb2ZfZGV0ZXJtaW5hdGlvbil7dGFyZ2V0PSJfYmxhbmsifSAgIApbUmlkZ2UgcmVncmVzc2lvbl0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvVGlraG9ub3ZfcmVndWxhcml6YXRpb24pe3RhcmdldD0iX2JsYW5rIn0gIApbTGFUZVggbWF0aGVtYXRpY2FsIG5vdGF0aW9uXShodHRwczovL3d3dy5jYWx2aW4uZWR1L35ycHJ1aW0vY291cnNlcy9zMzQxL1MxNy9mcm9tLWNsYXNzL01hdGhpblJtZC5odG1sKXt0YXJnZXQ9Il9ibGFuayJ9ICAgCgpGb3IgbW9yZSBpbmZvcm1hdGlvbiBvbiBsaW5lYXIgcmVncmVzc2lvbiBzZWUgdGhpcyBbYm9va10oaHR0cHM6Ly9yYWZhbGFiLmdpdGh1Yi5pby9kc2Jvb2svbGluZWFyLW1vZGVscy5odG1sI2xpbmVhci1yZWdyZXNzaW9uLWluLXRoZS10aWR5dmVyc2Upe3RhcmdldD0iX2JsYW5rIn0gYW5kIHRoaXMgW2Nhc2Ugc3R1ZHldKGh0dHBzOi8vb3BlbmNhc2VzdHVkaWVzLmdpdGh1Yi5pby9vY3MtYnAtZGlldC8pe3RhcmdldD0iX2JsYW5rIn0uCgpGb3IgbW9yZSBpbmZvcm1hdGlvbiBvbiB0aGUgZGlmZmVyZW50IHR5cGVzIG9mIHBhbmVsIHJlZ3Jlc3Npb24gbW9kZWxzIHNlZSB0aGlzIFtib29rXShodHRwczovL2Jvb2tkb3duLm9yZy9jY29sb25lc2N1L1JQb0U0L3BhbmVsLWRhdGEtbW9kZWxzLmh0bWwpLCAgW2hlcmVdKGh0dHBzOi8vd3d3LmJhdWVyLnVoLmVkdS9yc3VzbWVsL3BoZC9lYzEtMTUucGRmKSwgYW5kIFtoZXJlXShodHRwczovL3NpdGVzLmdvb2dsZS5jb20vc2l0ZS9lY29ub21ldHJpY3NhY2FkZW15L2Vjb25vbWV0cmljcy1tb2RlbHMvcGFuZWwtZGF0YS1tb2RlbHMpLgoKRm9yIG1vcmUgaW5mb3JtYXRpb24gb24gaW1wbGVtZW50aW5nIHBhbmVsIHJlZ3Jlc3Npb24gaW4gUiB1c2luZyB0aGUgYHBsbWAgcGFja2FnZSwgc2VlIFtoZXJlXShodHRwczovL2NyYW4uci1wcm9qZWN0Lm9yZy93ZWIvcGFja2FnZXMvcGxtL3ZpZ25ldHRlcy9wbG1QYWNrYWdlLmh0bWwpe3RhcmdldD0iX2JsYW5rIn0gIGFuZCBbaGVyZV0oaHR0cDovL3d3dy5wcmluY2V0b24uZWR1L35vdG9ycmVzL1BhbmVsMTAxUi5wZGYpe3RhcmdldD0iX2JsYW5rIn0uCgpGb3IgbW9yZSBpbmZvcm1hdGlvbiBvbiBtdWx0aWNvbGxpbmVhcml0eSBhbmQgVklGLCBzZWUgdGhpcyBbYXJ0aWNsZV0oaHR0cHM6Ly9saW5rLnNwcmluZ2VyLmNvbS9jb250ZW50L3BkZi8xMC4xMDA3L3MxMTEzNS0wMDYtOTAxOC02LnBkZil7dGFyZ2V0PSJfYmxhbmsifS5ET0kgMTAuMTAwNy9zMTExMzUtMDA2LTkwMTgtNgoKVGhlIGFydGljbGVzIHVzZWQgdG8gbW90aXZhdGUgdGhpcyBjYXNlIHN0dWR5IGFyZTogICAKW011c3RhcmQgYW5kIExvdHRdKGh0dHBzOi8vY2hpY2Fnb3VuYm91bmQudWNoaWNhZ28uZWR1L2NnaS92aWV3Y29udGVudC5jZ2k/YXJ0aWNsZT0xMTUwJmNvbnRleHQ9bGF3X2FuZF9lY29ub21pY3Mpe3RhcmdldD0iX2JsYW5rIn0gIApbRG9ub2h1ZSwgZXQgYWwuXShodHRwczovL3d3dy5uYmVyLm9yZy9wYXBlcnMvdzIzNTEwLnBkZil7dGFyZ2V0PSJfYmxhbmsifSAgICAgCltTZWUgaGVyZSBmb3IgYSBsaXN0IG9mIHN0dWRpZXMgb24gdGhpcyB0b3BpYyBdKGh0dHBzOi8vZW4ud2lraXBlZGlhLm9yZy93aWtpL01vcmVfR3VucyxfTGVzc19DcmltZSl7dGFyZ2V0PSJfYmxhbmsifSAgCgpGb3IgdHV0b3JpYWxzIGFib3V0IHVzaW5nIGZpeGVkIGVmZmVjdCBtb2RlbHMgaW4gcGFuZWwgYW5hbHlzaXMgd2l0aCB0aGUgYHBsbWAgcGFja2FnZSwgc2VlIFtoZXJlXShodHRwczovL3JwdWJzLmNvbS9yc2xibGlzcy9maXhlZF9lZmZlY3RzKSAKYW5kIFtoZXJlXShodHRwOi8va2FydGh1ci5vcmcvMjAxOS9pbXBsZW1lbnRpbmctZml4ZWQtZWZmZWN0cy1wYW5lbC1tb2RlbHMtaW4tci5odG1sKSAgCgo8dT4qKlBhY2thZ2VzIHVzZWQgaW4gdGhpcyBjYXNlIHN0dWR5OioqIDwvdT4gIAoKICBQYWNrYWdlICAgfCBVc2UgaW4gdGhpcyBjYXNlIHN0dWR5ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCi0tLS0tLS0tLS0gfC0tLS0tLS0tLS0tLS0KW2hlcmVdKGh0dHBzOi8vZ2l0aHViLmNvbS9qZW5ueWJjL2hlcmVfaGVyZSl7dGFyZ2V0PSJfYmxhbmsifSAgICAgICB8IHRvIGVhc2lseSBsb2FkIGFuZCBzYXZlIGRhdGEKW2RwbHlyXShodHRwczovL2RwbHlyLnRpZHl2ZXJzZS5vcmcvKXt0YXJnZXQ9Il9ibGFuayJ9ICAgICAgfCB0byBhcnJhbmdlL2ZpbHRlci9zZWxlY3QvY29tcGFyZSBzcGVjaWZpYyBzdWJzZXRzIG9mIHRoZSBkYXRhICAKW21hZ3JpdHRyXShodHRwczovL2NyYW4uci1wcm9qZWN0Lm9yZy93ZWIvcGFja2FnZXMvbWFncml0dHIvdmlnbmV0dGVzL21hZ3JpdHRyLmh0bWwpe3RhcmdldD0iX2JsYW5rIn0gfCB0byB1c2UgdGhlIGNvbXBvdW5kIGFzc2lnbm1lbnQgcGlwZSBvcGVyYXRvciBgJTw+JWAKW3B1cnJyXShodHRwczovL3B1cnJyLnRpZHl2ZXJzZS5vcmcvKXt0YXJnZXQ9Il9ibGFuayJ9ICAgfCB0byBpbXBvcnQgdGhlIGRhdGEgaW4gYWxsIHRoZSBkaWZmZXJlbnQgZXhjZWwgYW5kIGNzdiBmaWxlcyBlZmZpY2llbnRseSAgClt0aWJibGVdKGh0dHBzOi8vdGliYmxlLnRpZHl2ZXJzZS5vcmcvKXt0YXJnZXQ9Il9ibGFuayJ9ICAgICB8IHRvIGNyZWF0ZSBkYXRhIG9iamVjdHMgdGhhdCB3ZSBjYW4gbWFuaXB1bGF0ZSB3aXRoIGBkcGx5cmAvYHN0cmluZ3JgL2B0aWR5cmAvYHB1cnJyYCAgCltnZ3Bsb3QyXShodHRwczovL2dncGxvdDIudGlkeXZlcnNlLm9yZy8pe3RhcmdldD0iX2JsYW5rIn0gICAgIHwgdG8gY3JlYXRlIHBsb3RzICAgCltnZ3JlcGVsXShodHRwczovL2NyYW4uci1wcm9qZWN0Lm9yZy93ZWIvcGFja2FnZXMvZ2dyZXBlbC92aWduZXR0ZXMvZ2dyZXBlbC5odG1sKXt0YXJnZXQ9Il9ibGFuayJ9ICAgIHwgdG8gYWxsb3cgbGFiZWxzIGluIGZpZ3VyZXMgbm90IHRvIG92ZXJsYXAgIApbcGxtXShodHRwczovL2NyYW4uci1wcm9qZWN0Lm9yZy93ZWIvcGFja2FnZXMvcGxtL3ZpZ25ldHRlcy9wbG1QYWNrYWdlLmh0bWwpe3RhcmdldD0iX2JsYW5rIn0gfCB0byB3b3JrIHdpdGggcGFuZWwgZGF0YSBmaXR0aW5nIGZpeGVkIGVmZmVjdHMgYW5kIGxpbmVhciByZWdyZXNzaW9uIG1vZGVscyAgClticm9vbV0oaHR0cHM6Ly9jcmFuLnItcHJvamVjdC5vcmcvd2ViL3BhY2thZ2VzL2Jyb29tL3ZpZ25ldHRlcy9icm9vbS5odG1sKXt0YXJnZXQ9Il9ibGFuayJ9IHwgdG8gY3JlYXRlIG5pY2VseSBmb3JtYXR0ZWQgbW9kZWwgb3V0cHV0ICAKW0dHYWxseV0oaHR0cHM6Ly9naXRodWIuY29tL2dnb2JpL2dnYWxseSl7dGFyZ2V0PSJfYmxhbmsifSB8IHRvIGV4dGVuZCBnZ3Bsb3QyIGZ1bmN0aW9uYWxpdHkgdG8gZWFzaWx5IGNyZWF0ZSBtb3JlIGNvbXBsZXggcGxvdHMgIApbZ2djb3JycGxvdF0oaHR0cHM6Ly93d3cucmRvY3VtZW50YXRpb24ub3JnL3BhY2thZ2VzL2dnY29ycnBsb3QvdmVyc2lvbnMvMC4xLjMpe3RhcmdldD0iX2JsYW5rIn0gfCB0byBlYXNpbHkgdmlzdWFsaXplIGEgY29ycmVsYXRpb24gbWF0cml4ICAKW3JzYW1wbGVdKGh0dHBzOi8vcnNhbXBsZS50aWR5bW9kZWxzLm9yZyl7dGFyZ2V0PSJfYmxhbmsifSB8IHRvIHNwbGl0IG91ciBzYW1wbGUgZm9yIHRoZSBzaW11bGF0aW9uIGFuYWx5c2lzICAgCltEVF0oaHR0cHM6Ly9yc3R1ZGlvLmdpdGh1Yi5pby9EVC8pe3RhcmdldD0iX2JsYW5rIn0gIHwgdG8gY3JlYXRlIGludGVyYWN0aXZlIGFuZCBzZWFyY2hhYmxlIHRhYmxlcyAgCltjYXJdKGh0dHBzOi8vY3Jhbi5yLXByb2plY3Qub3JnL3dlYi9wYWNrYWdlcy9jYXIvdmlnbmV0dGVzL2VtYmVkZGluZy5wZGYpe3RhcmdldD0iX2JsYW5rIn0gIHwgdG8gY2FsY3VsYXRlIFZJRiB2YWx1ZXMgb24gbGluZWFyIG1vZGVsIG91dHB1dCAgCltzdHJpbmdyXShodHRwczovL3N0cmluZ3IudGlkeXZlcnNlLm9yZy9hcnRpY2xlcy9zdHJpbmdyLmh0bWwpe3RhcmdldD0iX2JsYW5rIn0gICAgfCB0byBtYW5pcHVsYXRlIHRoZSBjaGFyYWN0ZXIgc3RyaW5ncyB3aXRoaW4gdGhlIGRhdGEgIApbY293cGxvdF0oaHR0cHM6Ly9jcmFuLnItcHJvamVjdC5vcmcvd2ViL3BhY2thZ2VzL2Nvd3Bsb3QvdmlnbmV0dGVzL2ludHJvZHVjdGlvbi5odG1sKXt0YXJnZXQ9Il9ibGFuayJ9IHwgdG8gYWxsb3cgcGxvdHMgdG8gYmUgY29tYmluZWQgCltsYXRleDJleHBdKGh0dHBzOi8vY3Jhbi5yLXByb2plY3Qub3JnL3dlYi9wYWNrYWdlcy9sYXRleDJleHAvdmlnbmV0dGVzL3VzaW5nLWxhdGV4MmV4cC5odG1sKXt0YXJnZXQ9Il9ibGFuayJ9IHwgdG8gY29udmVydCBsYXRleCBtYXRoIGZvcm11bGFzIHRvIFIncyBwbG90bWF0aCBleHByZXNzaW9ucyAgCgojIyAqKlNlc3Npb24gSW5mbyoqCioqKgoKYGBge3J9CnNlc3Npb25JbmZvKCkKYGBgCgoqKkVzdGltYXRlIG9mIFJNYXJrZG93biBDb21waWxhdGlvbiBUaW1lOiAqKgoKYGBge3IsIGVjaG89RkFMU0V9CnJtYXJrZG93bjo6OnBlcmZfdGltZXJfc3RvcCgicmVuZGVyIikKcHRzID0gcm1hcmtkb3duOjo6cGVyZl90aW1lcl9zdW1tYXJ5KCkKY2F0KCJBYm91dCIsIHJvdW5kKHB0cyR0aW1lWzFdLzEwMDAgKyA1KSwgIi0iLCByb3VuZChwdHMkdGltZVsxXS8xMDAwICsgMTUpLCJzZWNvbmRzIikKYGBgCgpUaGlzIGNvbXBpbGF0aW9uIHRpbWUgd2FzIG1lYXN1cmVkIG9uIGEgUEMgbWFjaGluZSBvcGVyYXRpbmcgb24gV2luZG93cyAxMC4gVGhpcyByYW5nZSBzaG91bGQgb25seSBiZSB1c2VkIGFzIGFuIGVzdGltYXRlIGFzIGNvbXBpbGF0aW9uIHRpbWUgd2lsbCB2YXJ5IHdpdGggZGlmZmVyZW50IG1hY2hpbmVzIGFuZCBvcGVyYXRpbmcgc3lzdGVtcy4gCgojIyAqKkFja25vd2xlZGdtZW50cyoqCioqKgoKV2Ugd291bGQgbGlrZSB0byBhY2tub3dsZWRnZSBbRGFuaWVsIFdlYnN0ZXJdKGh0dHBzOi8vd3d3Lmpoc3BoLmVkdS9mYWN1bHR5L2RpcmVjdG9yeS9wcm9maWxlLzczOS9kYW5pZWwtd2Vic3RlcikgZm9yIGFzc2lzdGluZyBpbiBmcmFtaW5nIHRoZSBtYWpvciBkaXJlY3Rpb24gb2YgdGhlIGNhc2Ugc3R1ZHkuIFdlIHdvdWxkIGFsc28gbGlrZSB0byB0aGFuayBbRWxpemFiZXRoIFN0dWFydF0oaHR0cHM6Ly93d3cuamhzcGguZWR1L2ZhY3VsdHkvZGlyZWN0b3J5L3Byb2ZpbGUvMTc5Mi9lbGl6YWJldGgtYS1zdHVhcnQpIGFuZCBbQWJvb3phciBIYWRhdmFuZF0oaHR0cHM6Ly93d3cubWluZXJ2YS5rZ2kuZWR1L3Blb3BsZS9hYm9vemFyLWhhZGF2YW5kLXBoZC1hc3Npc3RhbnQtcHJvZmVzc29yLWNvbXB1dGF0aW9uYWwtc2NpZW5jZXMvKSBhbmQgW0FsZXhhbmRlciBNY0NvdXJ0XShodHRwczovL3B1YmxpY2hlYWx0aC5qaHUuZWR1L2ZhY3VsdHkvMzc5NC9hbGV4YW5kZXItbWNjb3VydCkgZm9yIHJldmlld2luZyB0aGUgY2FzZSBzdHVkeS4gCgpXZSB3b3VsZCBsaWtlIHRvIGFja25vd2xlZGdlIFtNaWNoYWVsIEJyZXNob2NrXShodHRwczovL21icmVzaG9jay5naXRodWIuaW8vKSBmb3IgaGlzIGNvbnRyaWJ1dGlvbnMgdG8gdGhpcyBjYXNlIHN0dWR5IGFuZCBkZXZlbG9waW5nIHRoZSBgT0NTZGF0YWAgcGFja2FnZS4KCldlIHdvdWxkIGFsc28gbGlrZSB0byBhY2tub3dsZWRnZSB0aGUgW0Jsb29tYmVyZyBBbWVyaWNhbiBIZWFsdGggSW5pdGlhdGl2ZV0oaHR0cHM6Ly9hbWVyaWNhbmhlYWx0aC5qaHUuZWR1LykgZm9yIGZ1bmRpbmcgdGhpcyB3b3JrLiAKCjxzY3JpcHQgdHlwZT0ndGV4dC9qYXZhc2NyaXB0JyBpZD0nY2x1c3RybWFwcycgc3JjPScvL2Nkbi5jbHVzdHJtYXBzLmNvbS9tYXBfdjIuanM/Y2w9MDgwODA4Jnc9YSZ0PXR0JmQ9OW9LaDl0LTFxTTVZTDVuTGhUTWVNdm9RZ3d0amVJUXY2OXd1RGhZVWRhdyZjbz1mZmZmZmYmY21vPTNhY2MzYSZjbW49ZmY1MzUzJmN0PTgwODA4MCc+PC9zY3JpcHQ+Cg==

Open Case Studies: Influence of Multicollinearity on Measured Impact of Right-to-Carry Gun Laws

Motivation

[source]

[source]

[source]

*ML is abbreviated as LM in the source article

Main Question

Learning Objectives

Context

[source]

[source]

[source]

Limitations

What are the data?

Data Import and Wrangling

Demographic and population data

State FIPS codes

Police staffing data

Unemployment data

Poverty data

Violent crime

Right-to-carry data

[source]

*ML is abbreviated as LM in the source article

Data Exploration

Data Analysis

Panel Analysis

Donohue, et al.

Mustard and Lott

RTC coefficient comparison

Multicollinearity analysis

Correlation

Scatter plots

Heatmaps

Coefficient estimate instability

VIF

Data Visualization

Summary

Suggested Homework

Additional Information

Helpful Links

Session Info

Acknowledgments