Flex Data Engineer : Skill requirements Role 1: Data Engineer Allocation: 100% Role Description & Requirements: ● Minimum experience : 8-15 years ● In-depth knowledge and experience of GCP Data services : Bigquery / Dataproc / Composer / Pubsub / Dataflow / GCS/ BigTable ● Must have proficient experience in GCP Databases : Bigtable / Spanner / CloudSQL / AlloyDB ○ Solid understanding of relational database concepts and technologies such as SQL, MySQL, PostgreSQL, or Oracle. ○ Hands-on experience with other cloud platforms and services such as AWS RDS, or Azure SQL Database. ○ Experience with NoSQL databases such as MongoDB, Scylla, Cassandra, or DynamoDB is a plus. ○ Familiarity with database performance tuning, optimization, and troubleshooting techniques is a plus ● Strong working experience on one or more of the Bigdata/Hadoop distributions(or ecosystems) like, Cloudera/Hortonworks, MapR, Azure HDInsight, IBM Open platform, Kafka, Hive, Spark etc ● Good understanding of the following AWS Data services, Redshift, RDS, Athena or SQS/Kinesis ● Good understanding of Native and external tables, with different file formats : Avro, ORC, Parquet ● CI/CD pipelines for data workloads using Cloud Build, Artifact Registry, Terraform ● Data governance solutioning using GCP governance tooling (Dataplex, Data Catalog) Tools and languages experience Required Must have ● GCP Dataflow, Pub/Sub, Cloud Composer, Cloud Workflow, BigQuery, Cloud Run, CloudBuild ● Must have: programming knowledge and willingness to be hands-on - Python, Java ● Specialization on streaming such as PubSub or Kafka or equivalent Good to Have ● Experience in BigQuery or Presto or Equivalent ● Experience with open source ecosystem and distributions such as Hadoop, Spark, Cloudera/Hortonworks and frameworks and tech such as SPARK, Oozie, Kafka, HBASE ● Understanding and experience with NoSQL databases such as HBASE, MongoDB, Cassandra, ● Knowledge of cloud databases such as Spanner, BigTable, Cloud SQL, DB migrations Certification ● Good to have GCP Data engineering Please share any profiles matching these requirements.
