Missions Write a bot for this dataset: 'Financ...

Write a bot for this dataset: 'Finance Licences (Securities) - Israel Securities Authority (ISA)'


Your mission is to write a bot (some code) to transform a website into open data

This is the site in question: http://www.isa.gov.il/Mutual_Funds/Data%20on%20mutual%20funds/Pages/%D7%A8%D7%A9%D7%99%D7%9E%D7%AA-%D7%A7%D7%A8%D7%A0%D7%95%D7%AA-%D7%A0%D7%90%D7%9E%D7%A0%D7%95%D7%AA.aspx

This is one for people with technical skills - specifically coding in Ruby or Python (other languages are in the pipeline)


This is the website, that displays data below the search function in HTML.

img img

This is an example from the first entry to show the kind of data you should scrape. Extract the headings and associate them with company names and any other data you can find. Take a look at the example below:

{"category": "Mutual Funds List", "Fund name in Hebrew": "(00) MTF Tel Bond - linked", "Fund name in English": "MTF (00) Tel Bond Tsmudot", "No. Stock Exchange": "5116900", "State of the fund": "Active", "Name of Director": "Migdal Mutual Funds Ltd", "Trustee": "Almagor - Brightman Trusts Ltd", "Of the Manager": "0%", "Remuneration of the Trustee": "0.04%", "Track No": "Exempt", "Fund classification - Title": "Fixed Income - Corporate and conversion", "Fund classification - Headline": "Tel Bond Index", "Secondary Title": "null", "Exposure Profile": "00", "Type of fund - Fund units are": "null", "Redeemable on Sunday": "Yes", "Leveraged fund": "Not", "Tracking fund": "Yes", "Reverse exposure fund": "Not", "Fund fixed dates": "Not", "Currency price": "NIS", "Primary Currency": "NIS", "Address": "Saadia Gaon Tel Aviv 26 67 135", "Phone": "03-5194031", "Fax": "03-5194031", "Email": "kranot@msh.co.il"}


Here's how we suggest you go about it:

  1. Start by clicking 'Accept this mission' on this page. Don't worry, you can always give up if you can't finish it.
  2. You'll write the scraper using our "Turbot" framework. Head over to the Turbot website to and click "Start contributing" to read a getting started guide.
  3. If you have any questions, whether they are technical or about the data, get in touch and ask!
  4. When you think you've written a suitable bot, submit it for review using the Turbot command line tool.
  5. Once we've checked over the data, we'll either tell you if anything needs to be fixed or we'll accept the bot which means your mission will be complete!
Still not sure? Don't worry!

Whilst this does require you to be able to code, its probably not as hard as you think. Take a look at our example bots to get a feel for what's required.

Radio chatter

comments powered by Disqus