gil lu

about

I'm a data engineering graduate student with hands-on experience building end-to-end data pipelines, ETL workflows, and cloud-connected data systems. Skilled in Python, SQL, PySpark, and AWS, with a track record of designing data infrastructure and producing analytics products used in production.

I'm comfortable working across the full data stack, from ingestion and transformation to visualization and stakeholder delivery. Based in Brooklyn, NY.

Skills

Languages
Python (pandas, NumPy)SQLT-SQLRJavaScriptTypeScriptJava
Data Engineering & Cloud
dbtDagsterApache KafkaApache Spark (PySpark, Structured Streaming)Databricks (Unity Catalog)Delta LakeTerraformAWS (S3, Glue, Athena, Lambda, EC2, EMR, Aurora)Azure (Data Lake Storage, Data Factory)
Databases
PostgreSQLMySQLSQL Server (SSMS)SQLiteMongoDBFirebase (Firestore)Redis (Upstash)Neo4jSupabaseSnowflakeBigQueryRedshift
Analytics & Visualization
TableauTableau PrepPower BIGrafanaExcelAlteryxMatplotlibSeabornPlotlyscikit-learnstatsmodelsSciPyXGBoostLightGBMOptunaPyTorchTensorFlowYOLOv5Tesseract OCRGoogle Cloud VisionCUDA
Developer Tools & Frameworks
GitGitHub Actions (CI/CD)DockerDocker ComposeFlaskFastAPINode.jsExpress.jsReactNext.jsTailwind CSSElectronVercelOAuth 2.0

Certifications

  • IBM Data Analyst Professional Certificate

GitHub Activity