Parsing Every Congressional Stock Disclosure: Eighteen CronJobs and a Lot of Bad PDFs
Introduction Members of the US Congress are required to disclose their stock trades. The disclosures are public. They are also, as a dataset, close to unusable: the House publishes a ZIP of filings whose underlying documents are frequently scanned images, the Senate publishes through a separate system with its own quirks, tickers are inconsistent or absent, and neither chamber publishes anything you could call an API. Several commercial services solve this and sell the result. I wanted to see how much of it I could do myself, in-cluster, from the primary sources. ...