DevOps Engineer- AWS
Zürich, Switzerland
Posted: | 4d ago |
Location: | Zürich, Switzerland |
Job Ref: | BH-50663 |
Salary: | CHF750 - CHF850 per annum |
Expiry date: | 12/20/2024 |
For our customer a major technical services organisation we are seeking a DevOps Engineer.
Role is onsite in Zurich, 3-6 months initially
Project Scope:
Analysis of potential savings: Gathering statistics about current data, used storage tiers, and availability of original assets in source system
. De-duplication of data:
Implementation of scripts to verify and delete assets and object versions from S3-compatible Object Stores while tracking new asset location in relational metadata store, taking into account the costs create through API requests and lifecycle policy changes.
Running and monitoring script execution in parallel execution engine to perform de-duplication in reasonable time and cost-efficient manor.
Optimizing lifecycle policies for newly ingested data:
Adjust policies and necesary script to improve the costs for new data ingested into the system.
Project Duration:
Rough estimation would be one person working 60% - 100% for 3 month, with probably option for extend to 6 additional months.
Requirements: Extensive experience with AWS S3, in particular Object History, Cost optimizations and Storage Tiers, Lifecycle Policies, API Request Charges. Confident in implementing code in Python, version control in Git, querying and modifying relational databases through SQL
Good to have: Experience with fsspec Experience with frameworks for parallel execution (e.g. Dask, PySpark) Experience with SQLAlchemy Core and ORM Experience with large using and optimizing PostgreSQL / AWS Aurora instances Functional programming preferred way of writing Python Optimizing I/O performance using Python asyncio or rewriting critical functions in Rust
On-Site Presence: On-site presence is required for the duration of the project. Remote work will not be supported; the person must be available to work at the project location to ensure close collaboration with the team.
Role is onsite in Zurich, 3-6 months initially
Project Scope:
Analysis of potential savings: Gathering statistics about current data, used storage tiers, and availability of original assets in source system
. De-duplication of data:
Implementation of scripts to verify and delete assets and object versions from S3-compatible Object Stores while tracking new asset location in relational metadata store, taking into account the costs create through API requests and lifecycle policy changes.
Running and monitoring script execution in parallel execution engine to perform de-duplication in reasonable time and cost-efficient manor.
Optimizing lifecycle policies for newly ingested data:
Adjust policies and necesary script to improve the costs for new data ingested into the system.
Project Duration:
Rough estimation would be one person working 60% - 100% for 3 month, with probably option for extend to 6 additional months.
Requirements: Extensive experience with AWS S3, in particular Object History, Cost optimizations and Storage Tiers, Lifecycle Policies, API Request Charges. Confident in implementing code in Python, version control in Git, querying and modifying relational databases through SQL
Good to have: Experience with fsspec Experience with frameworks for parallel execution (e.g. Dask, PySpark) Experience with SQLAlchemy Core and ORM Experience with large using and optimizing PostgreSQL / AWS Aurora instances Functional programming preferred way of writing Python Optimizing I/O performance using Python asyncio or rewriting critical functions in Rust
On-Site Presence: On-site presence is required for the duration of the project. Remote work will not be supported; the person must be available to work at the project location to ensure close collaboration with the team.
Apply now
Contact:
Position:
Manager
Sector:
IT Infrastructure
Contact Email:
Telephone: