Country/Region:
IN
Requisition ID:
37300
Work Model:
Position Type:
Salary Range:
Location:
INDIA - BENGALURU - BIRLASOFT OFFICE
Title: Subcontractor
Description:
Area(s) of responsibility
Job Title: Pipeline Engineer – Rewrite & Deployment
Role Summary
Responsible for redesigning, rewriting, and deploying data/application pipelines to improve scalability, performance, and reliability. The role involves working closely with development, QA, DevOps, and business teams to ensure seamless delivery and deployment of modernized pipelines.
Key Responsibilities
- Analyze existing pipelines (ETL/data/process workflows) and identify gaps, inefficiencies, and improvement areas
- Design and rewrite pipelines using modern frameworks, tools, and best practices
- Optimize pipelines for performance, scalability, and fault tolerance
- Implement automated deployment processes using CI/CD pipelines
- Ensure version control, code quality, and proper documentation of rewritten pipelines
- Perform unit, integration, and system-level validation prior to deployment
- Collaborate with QA teams for validation and testing strategy alignment
- Monitor deployed pipelines and troubleshoot issues post-deployment
- Ensure compliance with security, governance, and data standards
- Work with stakeholders to plan cutover, migration, and rollback strategies
Required Skills & Experience
- Strong experience in pipeline development (ETL/data pipelines or application pipelines)
- Hands-on experience with tools such as Azure Data Factory, AWS Glue, Apache Airflow, Jenkins, or similar
- Proficiency in programming/scripting (Python, SQL, Shell, etc.)
- Experience with CI/CD tools and deployment automation
- Knowledge of cloud platforms (Azure/AWS/GCP)
- Strong debugging and performance optimization skills
- Familiarity with version control systems (Git)
- Understanding of DevOps and Agile methodologies
Preferred Qualifications
- Experience in pipeline migration or legacy system modernization
- Knowledge of containerization (Docker/Kubernetes)
- Exposure to monitoring/logging tools (Splunk, ELK, etc.)
- Experience working in distributed or global teams
Key Deliverables
- Fully rewritten and optimized pipelines
- Deployment-ready artifacts with CI/CD integration
- Reduced execution time / improved efficiency metrics
- Stable and monitored production deployment
Soft Skills
- Strong analytical and problem-solving skills
- Effective stakeholder communication
- Ability to handle multiple priorities and deadlines
- Ownership and accountability for end-to-end delivery