« Changelog History


krawler v0.4.3 Release Notes

Release Date: 2017-11-26 // over 5 years ago
    • ➕ Added ability to clear crawl queues by RequestId and Age, see Krawler#removeUrlsByRootPage
      🚚 and Krawler#removeUrlsByAge
    • ➕ Added config option to prevent crawler shutdown on empty queues
    • ➕ Added new single byte priority field to KrawlQueueEntry. Queues will always attempt to pop the lowest priority
      entry available. Priority can be assigned by overriding the Krawler#assignQueuePriorty method.
    • ⚡️ Update dependencies