James Mills
|
ce4b180576
|
Add Indexer.Size() and index_size metric
|
2021-02-07 21:12:49 +10:00 |
|
James Mills
|
c31fc9569f
|
Fix paging
|
2021-02-07 19:06:25 +10:00 |
|
James Mills
|
aee086ea34
|
Always scrape the first page
|
2021-02-07 18:49:31 +10:00 |
|
James Mills
|
b34ff5ade2
|
Remove useless log file
|
2021-02-06 08:13:38 +10:00 |
|
James Mills
|
36cdbd3fc5
|
Tidied up the UX a bit more
|
2021-02-03 21:29:18 +10:00 |
|
James Mills
|
ccfbd18b29
|
Add crawler_duration gauge and server_queries counter
|
2021-02-03 19:09:16 +10:00 |
|
James Mills
|
6dc7510e8f
|
Fixed pager
|
2021-02-03 01:56:56 +10:00 |
|
James Mills
|
382fd677c3
|
Remove superfluous notice on search results
|
2021-02-03 01:01:47 +10:00 |
|
James Mills
|
929c91af02
|
Improve cache view
|
2021-02-03 00:29:25 +10:00 |
|
James Mills
|
1708838316
|
Fixed cache template and handleer
|
2021-02-03 00:14:40 +10:00 |
|
James Mills
|
4e7b443feb
|
Code cleanup
|
2021-02-02 23:56:20 +10:00 |
|
James Mills
|
4c02ee5afc
|
Improve the search results view
|
2021-02-02 23:34:18 +10:00 |
|
James Mills
|
867b7b141d
|
Add quick bookmarklet
|
2021-02-02 17:26:12 +10:00 |
|
James Mills
|
e34e331101
|
Fix autofocus of input fields
|
2021-02-02 16:39:12 +10:00 |
|
James Mills
|
f47e9ba765
|
Fix crappy top notice that looks terrible on mobile
|
2021-02-02 15:44:00 +10:00 |
|
James Mills
|
383e6837ac
|
Add current search query to form and render search on index if q= is non-nil
|
2021-02-02 15:32:23 +10:00 |
|
James Mills
|
93cab4c1bc
|
Add opensearch endpoint and handler
|
2021-02-02 15:12:08 +10:00 |
|
James Mills
|
3923ff91ba
|
Add a /tasks endpoing and extra metdata to crawl tasks
|
2021-02-02 14:58:52 +10:00 |
|
James Mills
|
5cd5495622
|
Fix close links chan
|
2021-02-02 14:35:10 +10:00 |
|
James Mills
|
0e142fc233
|
Close links chan on crawl completion
|
2021-02-02 14:16:17 +10:00 |
|
James Mills
|
e1b0dd173f
|
Reduce dispatchers to 2 (by default)
|
2021-02-02 13:36:48 +10:00 |
|
James Mills
|
26df009e8f
|
Refactored crawler to use task dispatcher
|
2021-02-02 13:13:12 +10:00 |
|
James Mills
|
4970b16d61
|
Add better crawler metrics
|
2021-02-02 12:07:53 +10:00 |
|
James Mills
|
865e5b5750
|
Rejig Login to Partner Login and put banner at top
|
2021-02-02 11:24:49 +10:00 |
|
James Mills
|
0db2b08911
|
Minor UI tweak
|
2021-02-02 08:25:54 +10:00 |
|
James Mills
|
03236ffa1f
|
It works (but it sucks0
|
2021-02-02 08:23:42 +10:00 |
|
James Mills
|
beae6d03c1
|
Keep data dir in git
|
2021-02-02 08:02:34 +10:00 |
|
James Mills
|
82da9eee99
|
Fix bug creating cache dir
|
2021-02-02 08:01:21 +10:00 |
|
James Mills
|
4383b3125d
|
Add additional fields to search request
|
2021-02-02 00:18:45 +10:00 |
|
James Mills
|
8a1161cf77
|
Record and skip urls the crawler has seen before
|
2021-02-01 23:46:26 +10:00 |
|
James Mills
|
b398a3a709
|
Add working Crawler, Scraper and Indexer
|
2021-02-01 22:38:17 +10:00 |
|
James Mills
|
753aff61a1
|
Update deps
|
2021-02-01 21:26:02 +10:00 |
|
James Mills
|
b69b27eeed
|
Add crawler skeleton and add handler
|
2021-02-01 21:25:32 +10:00 |
|
James Mills
|
8c84ee8b3d
|
Add link crawler
|
2021-02-01 20:20:25 +10:00 |
|
James Mills
|
a7a928d7dc
|
Add login template
|
2021-01-31 15:17:26 +10:00 |
|
James Mills
|
4e7799b1a5
|
Add add template
|
2021-01-31 15:14:08 +10:00 |
|
James Mills
|
af1769a4a9
|
Fixed missing li
|
2021-01-31 15:01:57 +10:00 |
|
James Mills
|
7df4cd3046
|
Removed unused styles
|
2021-01-31 15:01:38 +10:00 |
|
James Mills
|
bd794790cc
|
Add missing icnos
|
2021-01-31 14:50:25 +10:00 |
|
James Mills
|
23e030a95d
|
Add types package
|
2021-01-30 23:50:26 +10:00 |
|
James Mills
|
9ae2a31cfa
|
Flresh out search template
|
2021-01-30 23:13:06 +10:00 |
|
James Mills
|
ca9ab1bf61
|
Update index/search template
|
2021-01-30 19:09:53 +10:00 |
|
James Mills
|
eb129f7937
|
Updated logo
|
2021-01-30 19:08:56 +10:00 |
|
James Mills
|
578e6abb9d
|
Update logo and favicon
|
2021-01-30 17:45:33 +10:00 |
|
James Mills
|
36c6049ff3
|
Fix base template
|
2021-01-30 17:26:18 +10:00 |
|
James Mills
|
61716aca3b
|
Delete spyda.db and add to .gitignore
|
2021-01-30 17:24:22 +10:00 |
|
James Mills
|
e7e3e14ca6
|
Add missing page template
|
2021-01-30 16:07:31 +10:00 |
|
James Mills
|
de4d0539f0
|
Fix base and index template
|
2021-01-30 16:06:56 +10:00 |
|
James Mills
|
a9be838b0b
|
Add /cache route
|
2021-01-30 15:58:18 +10:00 |
|
James Mills
|
c4f53a0be9
|
It builds!
|
2021-01-30 15:56:03 +10:00 |
|