James Mills 26df009e8f Refactored crawler to use task dispatcher

2021-02-02 13:13:12 +10:00

..

auth

Initial Codebase (untested)

2021-01-30 14:05:04 +10:00

pages

Replaced all other referenced to twt/twtxt

2021-01-30 15:06:58 +10:00

passwords

Initial Codebase (untested)

2021-01-30 14:05:04 +10:00

session

Initial Codebase (untested)

2021-01-30 14:05:04 +10:00

static

Rejig Login to Partner Login and put banner at top

2021-02-02 11:24:49 +10:00

templates

Rejig Login to Partner Login and put banner at top

2021-02-02 11:24:49 +10:00

api.go

More fixes

2021-01-30 15:26:24 +10:00

auth_handlers.go

Add password hadnelrs

2021-01-30 15:38:26 +10:00

base_task.go

Refactored crawler to use task dispatcher

2021-02-02 13:13:12 +10:00

bitcask_store.go

Record and skip urls the crawler has seen before

2021-02-01 23:46:26 +10:00

config.go

Replaced all other referenced to twt/twtxt

2021-01-30 15:06:58 +10:00

context.go

It works (but it sucks0

2021-02-02 08:23:42 +10:00

crawl_task.go

Refactored crawler to use task dispatcher

2021-02-02 13:13:12 +10:00

crawler.go

Refactored crawler to use task dispatcher

2021-02-02 13:13:12 +10:00

dispatcher_test.go

Refactored crawler to use task dispatcher

2021-02-02 13:13:12 +10:00

dispatcher.go

Refactored crawler to use task dispatcher

2021-02-02 13:13:12 +10:00

email.go

Add password hadnelrs

2021-01-30 15:38:26 +10:00

entry.go

Add working Crawler, Scraper and Indexer

2021-02-01 22:38:17 +10:00

errors.go

Initial Codebase (untested)

2021-01-30 14:05:04 +10:00

func_task.go

Refactored crawler to use task dispatcher

2021-02-02 13:13:12 +10:00

handlers.go

It works (but it sucks0

2021-02-02 08:23:42 +10:00

indexer.go

Add additional fields to search request

2021-02-02 00:18:45 +10:00

init.go

Initial Codebase (untested)

2021-01-30 14:05:04 +10:00

jobs.go

More fixes

2021-01-30 15:26:24 +10:00

links.go

Add working Crawler, Scraper and Indexer

2021-02-01 22:38:17 +10:00

manage_handlers.go

It builds!

2021-01-30 15:56:03 +10:00

models.go

Record and skip urls the crawler has seen before

2021-02-01 23:46:26 +10:00

options.go

It builds!

2021-01-30 15:56:03 +10:00

passwd_handlers.go

Add mixxing Recovery field on User object

2021-01-30 15:42:27 +10:00

results.go

It works (but it sucks0

2021-02-02 08:23:42 +10:00

robots.go

Replaced all other referenced to twt/twtxt

2021-01-30 15:06:58 +10:00

router.go

Add missing router module

2021-01-30 15:16:44 +10:00

scraper.go

Fix bug creating cache dir

2021-02-02 08:01:21 +10:00

server.go

Refactored crawler to use task dispatcher

2021-02-02 13:13:12 +10:00

session_store.go

Initial Codebase (untested)

2021-01-30 14:05:04 +10:00

stats.go

Initial Codebase (untested)

2021-01-30 14:05:04 +10:00

store.go

Record and skip urls the crawler has seen before

2021-02-01 23:46:26 +10:00

support_handlers.go

It builds!

2021-01-30 15:56:03 +10:00

task_handler.go

Refactored crawler to use task dispatcher

2021-02-02 13:13:12 +10:00

task.go

Refactored crawler to use task dispatcher

2021-02-02 13:13:12 +10:00

templates.go

Add missing handlers

2021-01-30 15:30:48 +10:00

ttlcache.go

Add missing TTLCache

2021-01-30 15:39:16 +10:00

utils.go

Refactored crawler to use task dispatcher

2021-02-02 13:13:12 +10:00

worker.go

Refactored crawler to use task dispatcher

2021-02-02 13:13:12 +10:00