5 Commits

Author	SHA1	Message	Date
James Mills	aee086ea34	Always scrape the first page	2021-02-07 18:49:31 +10:00
James Mills	5cd5495622	Fix close links chan	2021-02-02 14:35:10 +10:00
James Mills	0e142fc233	Close links chan on crawl completion	2021-02-02 14:16:17 +10:00
James Mills	b398a3a709	Add working Crawler, Scraper and Indexer	2021-02-01 22:38:17 +10:00
James Mills	8c84ee8b3d	Add link crawler	2021-02-01 20:20:25 +10:00