5 Commits

Author	SHA1	Message	Date
James Mills	26df009e8f	Refactored crawler to use task dispatcher	2021-02-02 13:13:12 +10:00
James Mills	8a1161cf77	Record and skip urls the crawler has seen before	2021-02-01 23:46:26 +10:00
James Mills	c4f53a0be9	It builds!	2021-01-30 15:56:03 +10:00
James Mills	3362aebbc1	Add missing NormalizeUsernaem	2021-01-30 15:40:02 +10:00
James Mills	4529ea3196	Initial Codebase (untested)	2021-01-30 14:05:04 +10:00