Web Scraping Specialist

The Role

The Role.

We are seeking a Web Scraping Specialist who is proficient and brings significant experience in data extraction and web scraping techniques. You will join a small, specialized team and lead efforts to gather and analyze data, optimize scraping processes, and support our vision for a future where Grass plays a crucial role in transforming internet data accessibility.

• Demonstrated ability to extract data from complex websites with minimal supervision, with a portfolio or examples of past projects.
• Knowledge of asynchronous programming, multithreading, and distributed scraping.
• Experience with No
  
  SQL databases (Mongo
  
  DB, Cassandra), capable of designing efficient storage solutions and managing data integrity.
• Ability to apply machine learning algorithms for data cleaning, categorization, or predictive analysis adds significant value.
• Experience with cloud services (AWS, Google Cloud, Azure) for deploying and managing scraping jobs at scale.
• Active participation in open-source projects related to web scraping, data processing, or similar fields.

• Write, test, and refine code that extracts data from various online sources, ensuring reliability and efficiency.
• Perform data retrieval tasks, handling complexities such as pagination and dynamic content loaded with AJAX.
• Clean and format extracted data, ensuring it meets quality standards for further analysis or processing.
• Database management:
  Store and manage the scraped data in appropriate databases, optimizing for access speed and data integrity.
• Regularly monitor the scraping processes, identify and resolve any issues to maintain continuous data flow.

• Opportunity. We are at at the forefront of developing a web-scale crawler and knowledge graph that allows ordinary people to participate in the process, and share in the benefits of AI development.
• Culture. We’re a lean team working together to achieve a very ambitious goal of improving access to public web data and distributing the value of AI to the people. We prioritize low ego and high output.
• Compensation. You’ll receive a competitive salary and equity package.