Data and calculations for the New Zealand Herald’s investigation into unresolved sexual assaults.
The investigation examines the increasing number of sexual assaults in New Zealand that are not being resolved.
Four charts were produced for the article. The sources and analysis used to generate the data for these charts in included below. The data extracts used are available for download and the analysis has been done in R using tidyverse libraries.
The charts where produced using Semiotic and exported into Adobe Illustrator for fine tuning.
Unresolved sexual assaults over time. This time series needs to be used with care as the NZ police data collection system was changed in 2014. The charts attempt to highlight this difference to readers while still showing a meaningful time series.
Under the new data collection system crimes are treated as being unresolved if there has been no outcome after 180 days. Using this metric 2016 is the last full calendar year it makes sense to look at.
Data on aggravated sexual assaults is publically available, but under the new reporting system data on rape is not publically available. The NZ Herald acquired this data via an OIA request. Aggravted sexual assault, rape (male vs female over 16), and serious assualt causing injury are compared.
Variations the rate of unresolved aggravated sexual assaults between police districts in 2016.
Data on the outcome of police investigations in New Zealand comes from two sources:
The calculations below attempt to merge the two series of data in a meaningful way.
Steps to download the data:
Default selection also includes
For comparison serious assault causing injury data was also downloaded from StatsNZ and is here
This analysis only considers sexual assaults against adults, therefore all offences against need to be removed.
stats_data <- read.csv('TABLECODE7405_Data_4257896b-fd89-4dac-bf13-b591c410f76f.csv')
offences <-read.csv('aggrv_offences.txt')
kable(stats_data %>% group_by(Offence) %>%
summarise() %>%
mutate(Included = ifelse(Offence %in% offences$Offence, 'Yes', 'No')) %>%
arrange(desc(Included),Offence))
Offence | Included |
---|---|
Anal Intercourse With Any Severely Subnormal Person | Yes |
Assault Intent Commit Sex Connect-Spouse | Yes |
Assault Intent Commit Sexual Connection Female Over 16 | Yes |
Assault With Intent To Commit Rape Female Over 16 | Yes |
Assault With Intent To Commit Rape Spouse | Yes |
Atmpt Sex Exploitation Significant Impairment | Yes |
Attempt To Rape Female Over 16 | Yes |
Attempt To Rape Spouse | Yes |
Attempted Unlawful Sexual Connection Female Over 16 | Yes |
Attempted Unlawful Sexual Connection Male Over 16 | Yes |
Attempted Unlawful Sexual Connection Spouse | Yes |
Attmpt Sex Connection Dependent Family Member | Yes |
Attmpt Sex Intercourse Child Under Care/Protection 16-20 | Yes |
Attmpt Sex Intercourse Severely Subnormal Female Over 16 | Yes |
Brother Incest Sister Over 16 | Yes |
Does Indecent Act-Person Significant Impairment | Yes |
Husband Rapes Wife | Yes |
Indecent Act On Dependent Family Member | Yes |
Induce Indecent Act | Yes |
Induce Sexual Connection | Yes |
Inducing Sexual Connection - Female Over 16 | Yes |
Inducing Sexual Intercourse Pretence Of Marriage | Yes |
Male Rapes Female (No Weapon) | Yes |
Males Rapes Female Over 16 | Yes |
Other Assault With Intent To Commit Sexual Violation | Yes |
Other Attempt To Commit Sexual Violation Offence | Yes |
Other Incest | Yes |
Other Indecency (Male-Female) | Yes |
Other Indecent Assault | Yes |
Other Indecent Assaults | Yes |
Other Inducing Sexual Connection Offences | Yes |
Other Inducing/Permitting Indecent Act | Yes |
Other Sexual Offences Against Male Victim | Yes |
Other Sexual Violation Offences | Yes |
Other Unlawful Sexual Intercourse | Yes |
Parent Incest Child Over 16 | Yes |
Sex Exploitation Significant Impairment | Yes |
Sexual Connection Dependent Family Member | Yes |
Sexual Intercourse With Child Under Care/Protection 16-20 | Yes |
Sexual Intercourse With Severely Subnormal Female Over 16 | Yes |
Unlawful Sexual Connection Female Over 16 | Yes |
Unlawful Sexual Connection Male Over 16 | Yes |
Unlawful Sexual Connection With Spouse | Yes |
Anal Intercourse With Any Person Under 16 | No |
Assault Intent Commit Sex Connection Female 12-16 | No |
Assault Intent Commit Sex Connection Female Under 12 | No |
Assault With Intent To Commit Rape Female 12-16 | No |
Assault With Intent To Commit Rape Female Under 12 | No |
Atmpt Sex Connection With Child Under 12 | No |
Atmpt Sex Connection With Person 12 - 16 | No |
Attempt To Rape Female 12-16 | No |
Attempt To Rape Female Under 12 | No |
Attempted Sexual Intercourse Female 12-16 | No |
Attempted Sexual Intercourse Female Under 12 | No |
Attempted Unlawful Sexual Connection Female 12-16 | No |
Attempted Unlawful Sexual Connection Female Under 12 | No |
Attempted Unlawful Sexual Connection Male 12-16 | No |
Attempted Unlawful Sexual Connection Male Under 12 | No |
Attmpt Sex Intercourse Child Under Care/Protection 12-16 | No |
Attmpt Sex Intercourse Severely Subnormal Female 12-16 | No |
Brother Incest Sister 12-16 | No |
Brother Incest Sister Under 12 | No |
Does Indecent Act Female With Boy 12 - 16 | No |
Does Indecent Act Female With Boy Under 12 | No |
Does Indecent Act Male With Girl 12-16 | No |
Does Indecent Act Male With Girl Under 12 | No |
Does Indecent Act Upon Girl 12-16 | No |
Does Indecent Act Upon Girl Under 12 | No |
Does Indecent Act With/Upon Boy 12-16 | No |
Does Indecent Act With/Upon Boy Under 12 | No |
Female Indecently Assaults Girl 12-16 | No |
Female Indecently Assaults Girl Under 12 | No |
Indecent Assault On Boy 12-16 | No |
Indecent Assault On Boy Under 12 | No |
Indecently Assaults Female 12-16 | No |
Indecently Assaults Female Under 12 | No |
Induce Indecent Act Girl 12-16 | No |
Induce Indecent Act Girl Under 12 | No |
Induce/Permit Boy 12-16 To Do Indecent Act | No |
Induce/Permit Boy Under 12 To Do Indecent Act | No |
Inducing Sexual Connection - Female 12-16 | No |
Inducing Sexual Connection - Female Under 12 | No |
Males Rapes Female 12-16 | No |
Males Rapes Female Under 12 | No |
Other Attmpt Sex Intercourse Offence Child Care/Protection | No |
Parent Incest Child 12-16 | No |
Parent Incest Child Under 12 | No |
Permit Indecent Act Girl 12-16 | No |
Permit Indecent Act Girl Under 12 | No |
Permits Indecent Act Male With Girl 12-16 | No |
Permits Indecent Act Male With Girl Under 12 | No |
Sexual Conduct With Child Outside N.Z. | No |
Sexual Connection With Child Under 12 | No |
Sexual Connection With Young Person 12 - 16 | No |
Sexual Intercourse With Child Under Care/Protection 12-16 | No |
Sexual Intercourse With Child Under Care/Protection Under 12 | No |
Sexual Intercourse With Female 12-16 | No |
Sexual Intercourse With Female Under 12 | No |
Sexual Intercourse With Severely Subnormal Female 12-16 | No |
Sexual Intercourse With Severely Subnormal Female Under 12 | No |
Unlawful Sexual Connection Female 12-16 | No |
Unlawful Sexual Connection Female Under 12 | No |
Unlawful Sexual Connection Male 12-16 | No |
Unlawful Sexual Connection Male Under 12 | No |
statsdata_adult <- stats_data %>%
filter(Offence %in% offences$Offence & Location != 'Total New Zealand')
Data downloaded from policedata.nz Victimisations demographics report. Only selected Aggravated Sexual Assault.
The following filters are applied to the data:
Download aggravated sexual assaults extract and serious assault causing injury extract. The serious assaults extract was extracted in a similar way to the aggravated sexual assaults data.
children <- c("0 - 4 years inclusive","5 - 9 years inclusive","10 - 14 years inclusive")
remove <- c("No crime", "Withdrawn by victim")
resolved <- c("Court action, n.f.d.", "Formal caution or formal warning", "Informal warning or informal caution", "Non-court action, n.f.d.")
policedata <- read.csv('victimisations.csv', sep='\t')
policedata_adult <- policedata %>%
filter(!(Age.Group.5Yr.Band %in% children)) %>%
filter(!(Outcome.Of.Investigation.180 %in% remove)) %>%
mutate(date = as.Date(Year.Month, format='%d/%m/%Y')) %>%
mutate(
Year = case_when(
date >= '2015-01-01' & date < '2016-01-01' ~ as.integer(2015),
date >= '2016-01-01' & date < '2017-01-01' ~ as.integer(2016),
date >= '2017-01-01' & date < '2017-08-31' ~ as.integer(2017),
TRUE ~ as.integer(-1))) %>%
filter(Year != -1)
assaults_police <- read.csv('assaults.csv', sep='\t') %>%
filter(!(Age.Group.5Yr.Band %in% children)) %>%
filter(!(Outcome.Of.Investigation.180 %in% remove)) %>%
mutate(date = as.Date(Year.Month, format='%d/%m/%Y')) %>%
mutate(
Year = case_when(
date >= '2015-01-01' & date < '2016-01-01' ~ as.integer(2015),
date >= '2016-01-01' & date < '2017-01-01' ~ as.integer(2016),
date >= '2017-01-01' & date < '2017-08-31' ~ as.integer(2017),
TRUE ~ as.integer(-1))) %>%
filter(Year != -1)
assaults_police_agg <- assaults_police %>%
group_by(Year) %>%
summarise(
Total = sum(Victimisations),
Resolved = sum(Victimisations[Outcome.Of.Investigation.180 %in% resolved])
) %>%
mutate(Rate = round((Total - Resolved) / Total * 100,1),
Unresolved = Total - Resolved,
Source = 'police')
The data released on policedata.nz does not include the specific offence male rapes female over 16. This data was requested from the police via an OIA
oia <- readxl::read_xlsx('oia.xlsx', range="Victimisations!A2:D71") %>%
rename(Area=X__1, Outcome=X__2) %>%
fill(Area) %>%
mutate(`2015`=as.integer(`2015`)) %>%
mutate(`2016`=as.integer(`2016`)) %>%
gather(Year, Count, -Area, -Outcome) %>%
filter(Outcome != 'Sub-total')
oia_agg <- oia %>%
filter(Outcome != 'No crime' & !is.na(Count)) %>%
group_by(Year) %>%
summarise(Total = sum(Count),
Resolved = sum(Count[Outcome == 'Court action' | Outcome == 'Non-court action n.f.d.'])) %>%
mutate(Unresolved = Total - Resolved, Rate = round((Total - Resolved) / Total * 100,1), Source='oia', Year=as.integer(Year))
kable(oia_agg)
Year | Total | Resolved | Unresolved | Rate | Source |
---|---|---|---|---|---|
2015 | 715 | 133 | 582 | 81.4 | oia |
2016 | 745 | 108 | 637 | 85.5 | oia |
The following table shows which investigation outcomes after 180 days are considered resolved
kable(policedata_adult %>%
group_by(Outcome.Of.Investigation.180) %>%
summarise() %>%
mutate(Resolved = ifelse(Outcome.Of.Investigation.180 %in% resolved, 'Yes', 'No')) %>%
arrange(desc(Resolved),Outcome.Of.Investigation.180))
Outcome.Of.Investigation.180 | Resolved |
---|---|
Court action, n.f.d. | Yes |
Formal caution or formal warning | Yes |
Informal warning or informal caution | Yes |
Non-court action, n.f.d. | Yes |
Investigation continuing | No |
Investigation pending or suspended | No |
No offender proceeded against, n.f.d. | No |
No outcome at this time | No |
No proceeding - initial attendance only | No |
Not pursued - Police discretion | No |
Transferred to another jurisdiction | No |
Unable to proceed | No |
Unresolved after investigation | No |
statsdata_adult_agg <- statsdata_adult %>%
group_by(Year) %>%
summarise(Resolved = sum(Value[Measure=='Resolved']),
Total= sum(Value[Measure=='Recorded'])) %>%
mutate(Rate = round((Total - Resolved)/Total*100,1),
Unresolved = Total - Resolved,
Source = 'stats')
policedata_adult_agg <- policedata_adult %>%
group_by(Year) %>%
summarise(
Total = sum(Victimisations),
Resolved = sum(Victimisations[Outcome.Of.Investigation.180 %in% resolved])
) %>%
mutate(Rate = round((Total - Resolved) / Total * 100,1),
Unresolved = Total - Resolved,
Source = 'police')
combined <- union(policedata_adult_agg, statsdata_adult_agg) %>% arrange(Year)
exp1 <- combined %>% select(Year, Rate, Source, Total)
write.csv(exp1, file='aggravated-over-time.csv',
row.names=FALSE)
list(exp1 %>% filter(Year <= 2014),
exp1 %>% filter(Year > 2014),
exp1 %>% filter(Year == 2014 | Year == 2015)
) %>%
toJSON() %>% write_lines('../graphics/src/aggravated-over-time.json')
kable(exp1)
Year | Rate | Source | Total |
---|---|---|---|
1994 | 44.7 | stats | 730 |
1995 | 44.5 | stats | 661 |
1996 | 43.2 | stats | 711 |
1997 | 49.8 | stats | 603 |
1998 | 52.0 | stats | 627 |
1999 | 55.4 | stats | 576 |
2000 | 51.8 | stats | 625 |
2001 | 50.5 | stats | 604 |
2002 | 48.1 | stats | 642 |
2003 | 49.8 | stats | 639 |
2004 | 50.7 | stats | 647 |
2005 | 50.3 | stats | 720 |
2006 | 44.5 | stats | 827 |
2007 | 42.9 | stats | 776 |
2008 | 39.6 | stats | 793 |
2009 | 45.0 | stats | 822 |
2010 | 52.3 | stats | 789 |
2011 | 51.7 | stats | 867 |
2012 | 54.6 | stats | 892 |
2013 | 63.2 | stats | 1010 |
2014 | 71.3 | stats | 1024 |
2015 | 77.7 | police | 2736 |
2016 | 82.1 | police | 2963 |
2017 | 87.3 | police | 3147 |
stats_rape <- statsdata_adult %>% filter(Offence == 'Males Rapes Female Over 16') %>%
group_by(Year) %>%
summarise(Resolved = sum(Value[Measure=='Resolved']),
Total= sum(Value[Measure=='Recorded'])) %>%
mutate(Rate = round((Total - Resolved)/Total*100,1),
Unresolved = Total - Resolved,
Source = 'stats')
rape = union(stats_rape, oia_agg) %>% select(Year, Rate, Source, Total)
assault <- read.csv('TABLECODE7405_Data_093506f6-703c-4eb1-b327-3431ab5ba7cb.csv') %>%
group_by(Year) %>%
summarise(Resolved = sum(Value[Measure=='Resolved']),
Total= sum(Value[Measure=='Recorded'])) %>%
mutate(Rate = round((Total - Resolved)/Total*100,1),
Unresolved = Total - Resolved,
Source = 'stats')
list(exp1 %>% filter(Year <= 2014),
exp1 %>% filter(Year > 2014),
exp1 %>% filter(Year == 2014 | Year == 2015),
rape %>% filter(Year <= 2014),
rape %>% filter(Year > 2014),
rape %>% filter(Year == 2014 | Year == 2015),
assault %>% select(Year, Rate, Source, Total),
assaults_police_agg %>% select(Year, Rate, Source, Total),
union(assault, assaults_police_agg) %>% select(Year, Rate, Source, Total) %>% filter(Year == 2014 | Year == 2015)
) %>% toJSON() %>% write_lines('../graphics/src/aggravated-vs-rape.json')
p2016 <- policedata_adult %>% filter(Year == 2016) %>%
group_by(Police.District) %>%
summarise(
Total = sum(Victimisations),
Resolved = sum(Victimisations[Outcome.Of.Investigation.180 %in% resolved])
) %>%
mutate(Rate = round((Total - Resolved) / Total * 100,1),
Unresolved = Total - Resolved,
Source = 'police') %>%
arrange(Rate)
p2016 %>% select(Police.District, Rate) %>%
toJSON() %>% write_lines('../graphics/src/police-districts.json')