moth

2013-09-05 · writing

moth
in brightened space
circle
and yearn for shadow

Isoxya: free web crawler

Isoxya is a web crawler and data processing system, representing years of research into building a next-generation web crawler. It can process websites with tens of millions of pages, and extract and transform that data in myriad ways, including streaming data into Elasticsearch.