Use-of-force records in six-point type

This PDF contains Vancouver Police use-of-force incident records, released to journalists after a public records request. The catch: the font is tiny, the rows are mostly empty space, and there isn’t a single ruling line — so table detectors see a haze of sparse text instead of columns.

from natural_pdf import PDF

pdf = PDF("pdfs/use-of-force-raw.pdf")
page = pdf.pages[0]
page.show()

Find the column headers

The headers are the text at the very top of the page, which means they have the smallest y0. min() inside an attribute filter finds them without hardcoding a coordinate:

headers = page.find_all('text[y0=min()]')
headers.extract_each_text()

['OFFICER',
 'DATE OF REPORT',
 'DATE OF FORCE',
 'TIME OF FORCE',
 'PRECINCT',
 'CASE #',
 'SUBJECT PERSON',
 'SEX',
 'RACE',
 'ETHNICITY',
 'SUBJECT CONDITION LEADING TO FORCE',
 'REASON FOR USE OF FORCE',
 'OFFICER ACTIVITY PRIOR TO FORCE',
 'SUBJECT WAS INJURED',
 'OFFICER INJURIES',
 'SUBJECT INJURIES',
 'MENTAL HEALTH INDICATORS',
 'FORCE OPTIONS ATTEMPTED',
 'EFFECTIVE FORCE OPTIONS',
 'TREATMENT RECEIVED OFFICER',
 'TREATMENT RECEIVED SUBJECT',
 'SUPERVISOR NOTIFIED',
 'SUPERVISOR AT SCENE',
 'SUPERVISOR NAME',
 'CASE COUNT']

Fit guides between the columns

from_headers() drops a vertical boundary between each pair of headers, nudging each divider to where it crosses the least text below:

from natural_pdf.guides import Guides

guides = Guides(page)
guides.vertical.from_headers(headers)
guides.show()

With the columns established, extract. Rows are worked out from the text layout behind the scenes:

guides.extract_table().to_df()

	OFFICER	DATE OF REPORT	DATE OF FORCE	TIME OF FORCE	PRECINCT	CASE #	SUBJECT PERSON	SEX	RACE	ETHNICITY	...	SUBJECT INJURIES	MENTAL HEALTH INDICATORS	FORCE OPTIONS ATTEMPTED	EFFECTIVE FORCE OPTIONS	TREATMENT RECEIVED OFFICER	TREATMENT RECEIVED SUBJECT	SUPERVISOR NOTIFIED	SUPERVISOR AT SCENE	SUPERVISOR NAME	CASE COUNT
0	PFEIFER, TIM	12/2/2016	Dec-02-2016	22:30	WEST	23-2016-18335	SOLLERS, CECIL ARTHUR	M	U	H	...	NONE	<NA>	TASER, CAROTID RESTRAINT	CAROTID RESTRAINT	<NA>	EMS AT SCENE, HOSPITAL/RELEASED	YES	YES	SGT. GEDDRY / 1315	970
1	NICHOLSON, DUSTIN	12/3/2016	Dec-03-2016	6:45	WEST	23-2016-18348	GUTIERREZ, SAVANNAH D (DOB: Aug-14-1986)	F	W	N	...	NONE	<NA>	TAKEDOWNS	TAKEDOWNS	TREATMENT REFUSED	TREATMENT REFUSED	YES	NO	SGT MARTIN /	970
2	JENNINGS, ERIK	12/7/2016	Dec-07-2016	13:05	EAST	23-2016-18571	MCDONALD, ALEMAYOHU J APR-03-1996	M	B	U	...	NONE	<NA>	TAKEDOWNS	TAKEDOWNS	<NA>	<NA>	YES	NO	BURGARA / 1257	970
3	BATES, JUSTIN	12/13/2016	Dec-02-2016	22:30	WEST	23-2016-18335	SOLLERS, CECIL A (DOB: Sep-25-1989)	M	U	H	...	NONE	<NA>	POINT TASER	POINT TASER	<NA>	EMS AT SCENE, HOSPITAL/ADMITTED, HOSPITAL/RELE...	YES	YES	CPL PARDUE/1219	970
4	GEDDRY, BLAISE	12/13/2016	Dec-13-2016	2:35	WEST	23-2016-18818	RENFRO, JOHN W	M	W	U	...	NONE	MENTAL ILLNESS, ADMISSION TO FACILITY, PRIOR K...	TAKEDOWNS, HANDS/FEET	TAKEDOWNS, HANDS/FEET	<NA>	EMS AT SCENE, HOSPITAL/ADMITTED	YES	YES	GEDDRY 1315	970
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
293	PARDUE, BILL	1/19/2018	1/19/2018	1:24	WEST	23-2018-967	RAULS, COURTNEY A	F	B	U	...	NONE	<NA>	TAKEDOWNS	TAKEDOWNS	<NA>	<NA>	YES	YES	DUMAS/ 1543	970
294	MARBACH, NICHOLAS	1/20/2018	1/18/2018	23:20	EAST	23-2018-965	ELLIOT, DENNIS RICHARD	M	U	U	...	BRUISES	MENTAL ILLNESS, ADMISSION TO FACILITY	HANDS/FEET	HANDS/FEET	<NA>	EMS AT SCENE, HOSPITAL/ADMITTED	YES	YES	KREBS/DAVIS #1508/1230	970
295	MARBACH, NICHOLAS	1/20/2018	1/18/2018	23:21	EAST	23-2018-965	NAPOLEON, ALICE LANEY	F	I	U	...	NONE	<NA>	HANDS/FEET	HANDS/FEET	<NA>	EMS AT SCENE, SELF TREATMENT	YES	YES	KREBS/DAVIS #1508/1230	970
296	HEMSTOCK, STUART	1/23/2018	1/23/2018	8:45	EAST	23-2018-1191	KESSEL, JORDAN P	M	W	N	...	NONE	<NA>	CONTROL HOLDS CAUSING INJURY	CONTROL HOLDS CAUSING INJURY	<NA>	<NA>	YES	YES	HUBERTY/1214	970
297	HAMMOND, SCOTLAND	1/27/2018	1/27/2018	20:20	WEST	23-2018-1420	CONTRERAS, RICKY	M	W	U	...	NONE	<NA>	HANDS/FEET	HANDS/FEET	<NA>	EMS AT SCENE	YES	YES	D. KREBS / 1508	970

298 rows × 25 columns

Combine the results for every page

Give extract_table() a list of pages and the same guides are applied to each one. Repeated column headers on later pages are removed automatically:

df = guides.extract_table(pdf.pages).to_df()
print("You found", len(df), "rows")

df.tail()

You found 970 rows

	OFFICER	DATE OF REPORT	DATE OF FORCE	TIME OF FORCE	PRECINCT	CASE #	SUBJECT PERSON	SEX	RACE	ETHNICITY	...	SUBJECT INJURIES	MENTAL HEALTH INDICATORS	FORCE OPTIONS ATTEMPTED	EFFECTIVE FORCE OPTIONS	TREATMENT RECEIVED OFFICER	TREATMENT RECEIVED SUBJECT	SUPERVISOR NOTIFIED	SUPERVISOR AT SCENE	SUPERVISOR NAME	CASE COUNT
965	ALDRIDGE, SANDRA	7/30/2019	7/29/2019	2:50	VPD EAST	23-2019-12373	NICHOLAS CALLAWAY	<NA>	<NA>	<NA>	...	LACERATIONS	MENTAL ILLNESS, PRIOR KNOWLEDGE	HANDS/FEET	HANDS/FEET	<NA>	EMS AT SCENE, HOSPITAL/ADMITTED, HOSPITAL/RELE...	REATMENT REFUSED YES	YES	BARTON/1551	970
966	RIEDEL, BRANDON	7/30/2019	7/30/2019	3:00	EAST PRECINCT	23-2019-12373	CALLAWAY, NICHOLAS RICHARD	M	U	N	...	LACERATIONS	<NA>	<NA>	HANDS/FEET	<NA>	EMS AT SCENE, HOSPITAL/RELEASED	YES	YES	SERGEANT BARTON/1551	970
967	SCOTT, MICHAEL	7/30/2019	7/30/2019	2:51	EAST	23-2019-12373	CALLAWAY, NICHOLAS	M	U	N	...	NONE	<NA>	CONTROL HOLDS CAUSING COMPLAINT OF INJURY	CONTROL HOLDS CAUSING COMPLAINT OF INJURY	<NA>	EMS AT SCENE, HOSPITAL/RELEASED	YES	YES	BARTON / 1551	970
968	VIGIL, NICOLE	7/30/2019	7/30/2019	5 MIN	EAST	23-2019-12373	CALLAWAY, NICHOLAS	M	U	N	...	ABRASIONS, LACERATIONS	<NA>	<NA>	<NA>	<NA>	EMS AT SCENE, HOSPITAL/RELEASED	YES	YES	BARTON / 1551	970
969	GIBSON, AARON	8/1/2019	Jun-24-2019	21:55	WEST	23-2019-10327	BLEVINS, ANDREW S (DOB: Apr-20-1990)	M	W	U	...	NONE	<NA>	HANDS/FEET	HANDS/FEET	<NA>	<NA>	YES	YES	A GIBSON / 1154	970

5 rows × 25 columns

Four pages, one DataFrame. If a later page shifted its columns (it happens with re-run report exports), build a fresh Guides per page from that page’s own header row instead of reusing one set of x-positions.