Skip to content

Latest commit

 

History

History
9 lines (9 loc) · 425 Bytes

README.md

File metadata and controls

9 lines (9 loc) · 425 Bytes

crawler

##Intro Crawl the infomations of apps from different channel.MyCrawler exposes the restful interface to the users. Users send the trigger infomations(e.g.taksId,channelId and so on),then the crawler can perform perfectly the task and response the apps infos to the user. ##Fetures

  • based on the webmagic0.5.2
  • uniform rest api
  • concurrent crawling approach
  • simpified models
  • high-efficiency HTML parser JSOUP