A crawler framework. It covers the whole lifecycle of crawler: downloading, url management, content extraction and persistent. It can simply the development of a specific crawler.

Latest Versions

29 versions โ†’
VersionVulnerabilitiesUsagesDate
0.10.x
0.10.3
0
Apr 23, 2024
0.10.2
0
Apr 23, 2024
0.10.1
0
Apr 23, 2024
0.9.x
0.9.1
0
Sep 10, 2023
0.9.0
0
Jun 22, 2023
0.8.x
0.8.0
0
Nov 23, 2022
0.7.x
0.7.6
0
Oct 23, 2022
0.7.5
0
Jul 22, 2021
0.7.4
0
Oct 26, 2020
0.6.x
0.6.1
0
Jan 21, 2017
0.6.0
0
Dec 17, 2016
0.5.x
0.5.3
0
Jan 21, 2016
0.5.2
0
Jun 04, 2014
0.5.1
0
May 02, 2014
0.4.x
0.4.3
0
Mar 12, 2014
0.4.2
0
Dec 03, 2013
0.4.1
0
Nov 28, 2013
0.3.x
0.3.2
0
Sep 23, 2013
0.3.1
0
Sep 08, 2013
0.3.0
0
Sep 04, 2013
0.2.x
0.2.1
0
Aug 20, 2013
29 versions โ†’