Missions Write a bot for this dataset: 'Financ...

Write a bot for this dataset: 'Finance Licences - Pensions and Insurance Authority (Zambia)'

Briefing

Your mission is to write a bot (some code) to transform a website into open data

This is the site in question: http://www.pia.org.zm/registered-insurance-entities

This is one for people with technical skills - specifically coding in Ruby or Python (other languages are in the pipeline)

Example

This is the website, that contains links to PDFs with the data.

img

This is an example from the PDF, extract the headings and associate them with the company names. Here is some example output for the screenshot below:

{"category": "REGISTERED PENSION SCHEME/FUND ADMINISTRATORS", "company_name": "Alexander Forbes Financial Services (Z) Limited"}
{"category": "REGISTERED PENSION SCHEME/FUND ADMINISTRATORS", "company_name": "AON Zambia Pension Fund Administrators Limited"}
{"category": "REGISTERED PENSION SCHEME/FUND ADMINISTRATORS", "company_name": "Benefits Consulting Services Limited"}
...

img

The above screenshot for example also contains data that should not be targeted. The "REGISTERED PENSION SCHEMES" for example is not data about companies itself, so in this case it should not be part of the output.

Here's how we suggest you go about it:

  1. Start by clicking 'Accept this mission' on this page. Don't worry, you can always give up if you can't finish it.
  2. You'll write the scraper using our "Turbot" framework. Head over to the Turbot website to and click "Start contributing" to read a getting started guide.
  3. If you have any questions, whether they are technical or about the data, get in touch and ask!
  4. When you think you've written a suitable bot, submit it for review using the Turbot command line tool.
  5. Once we've checked over the data, we'll either tell you if anything needs to be fixed or we'll accept the bot which means your mission will be complete!
  6. In general, don't hesitate to contact the Google Group if you have any questions about the data output or post it in the Slack chat.
Notes

The scraper for this mission will involve creating a single dataset from multiple pages or documents, read the following instructions for guidance on how to approach this.

Still not sure? Don't worry! Whilst this does require you to be able to code, its probably not as hard as you think. Take a look at our example bots to get a feel for what's required.


Radio chatter

comments powered by Disqus