Quick Apply
- Develop and maintain data pipeline will be the primary responsibility
- Maintain, expand and improve our ETL process and computational infrastructure
- Working with different sources of data (Web crawling and data feed) to process and transform them during ETL process
- Ensure and sustain data quality, such as error detection and report generation
- Responsible of initiatives related to building, maintaining and orchestrating all the components in data platform
Sr./ Data Engineer
Taipei City, Taiwan
Data Engineer/Machine Learning
Full Time
What You'll Do / Responsibilities
Required Qualifications
- Familiar with search engine (elasticsearch or solr)
- Familiar with web crawling (scrapy, etc.)
- Experience in workflow management tool (airflow, nifi, etc.)
- Passion for software development and committed to delivering outstanding work
- Familiar with Linux operating system
- Good programming skill in Python
Preferred Qualifications
- Experience in massive data processing platform (hadoop, spark, etc.)
Salary
Negotiable
Language: