Machine Learning Data Engineer

Subhas Hati

Email: ad5p6t@r.postjobfree.com

Phone: 718-***-****

Location: Queens, New York

Green Card, Last 4-SSN: 7006

Summary:

Highly skilled Data Engineer with expertise in Python development, distributed Spark, Spark Batch Processing, data warehousing in advanced NoSQL datastores, and proficient in applying Machine Learning techniques to large datasets. Extensive experience in designing, implementing, and optimizing data pipelines for efficient data processing and analysis.

Education:

PhD in Computer Engineering, IIT Kharagpur, 1995-2001

MS in Computer Engineering, Jadavpur University, Calcutta, 1988-1990

BS in Electronics and Communication Engineering, Jadavpur University, Calcutta, 1983-1987

Skills:

Programming Languages: Python (expert), Java, JavaScript, C++, C

Big Data Technologies: Apache Spark, Hadoop, AWS Glue, AWS Databricks

Data Storage: NoSQL Datastores (e.g., MongoDB, Cassandra)

Machine Learning Frameworks: TensorFlow, Keras

Data Warehousing: ETL, Data Modeling, Data Warehousing Design

Other: Distributed Systems, DevOps, CI/CD

Experiences:

AMPCUS / Intel INC, USA (On Contract) March 2022 – Feb 2024

Sr Data Engineer III (Python & Pyspark)

Led development and optimization of Python-based ETL workflows, leveraging distributed Spark for efficient data processing.

Implemented Spark Batch Processing for large-scale data analytics tasks, ensuring high performance and reliability.

Migrated and transformed data from advanced NoSQL data stores to the Hadoop ecosystem for comprehensive data warehousing solutions.

Applied Machine Learning techniques for data analysis and predictive modeling, enhancing business insights and decision-making processes.

Amaze Technology, LLC, USA Feb 2021- March 2022

Senior Data Engineer (Python)

Designed and implemented ETL data pipelines using Python and AWS Databricks, facilitating seamless data processing and analysis.

Integrated CI/CD pipelines with GitHub and AWS API Gateway for automated deployment and monitoring of data pipelines.

Leveraged Spark for processing BIG DATA, optimizing performance and scalability of data processing tasks.

Videonetic Technology Pvt. LTD. Kolkata, India June 2014- Jan 2021

Senior Developer

Developed and deployed video segmentation algorithms using Python and machine learning techniques for video surveillance applications.

Implemented data storage solutions using advanced NoSQL data stores, ensuring efficient and scalable data management.

Collaborated with cross-functional teams to design and optimize data warehousing solutions for large-scale video datasets.

Post-Doctoral Fellow (2000-02)

University Joseph Fourier, Grenoble, France

Project Name: Art-Live sponsored by EU Technical Commission.

Development of Augmented Reality Environment.

Research Scientist/Senior Research Scientist (2002-2017)

Institute of Infocomm Research, Singapore

1.Head Pose Estimation of News Reader From A Digital Video.

2.Brain Image Classification using Deep Learning Technique.

3.Digital Video Segmentation using machine learning techniques.

GoogleScholar:

https://scholar.google.com/citations?user=_MRnnN0AAAAJ&hl=en&oi=ao

Journal Papers:

S. Hati and S. Sengupta, “Robust Camera Parameter Estimation using Genetic

Algorithm”, Pattern Recognition Letters, Vol. 22(3-4), pp 289-298, March 2001.

[Google Scholar: 51 Citations]

A. W. Reza, C. Eswaran and S. Hati, “Automatic Tracing of Optic Disc and Exudates from Color Fundus Images using Fixed and Variable Thresholds”, Journal of

Medical Systems, Vol. 32(2), No. 3, pp. 73-80, 2008. [Google Scholar: 115 Citations]

S. Hati and S. Sengupta, “Pose Estimation in Automated Visual Inspection using Artificial Neural Networks” Vol. 12, No. 6, pp 483-496, 2002.

S. Hati, “Estimation of Pose Parameters from a Set of Least Square Objective Functions”, Machine, Graphics and Vision, Vol. 17, No. 3, pp 299-312, 2008.

S. Hati, K. Chaudhury and A. Ibrahim, Pose Estimation in Automated Visual Inspection using GA”, International Journal of Neural System, Vol. 16, No. 6, pp 255-269, 2006.

M. D. Shabe and S. Hati, Decision Fusion Based on Voting Scheme for IR and Visible Face Recognition”, Machine Graphics and Vision, Vol. 17, No. 4, pp 403-424, 2008.

A. W. Reza, C. Eswaran and S. Hati, Diabetic Retinopathy: A Quadtree Based Blood Vessel Detection Algorithm using RGB Components in Fundus Images”, Journal of Medical Systems, Vol. 32, No. 2, pp. 147-155, 2008.

M. D. Shabe and S. Hati, Kernel Based Subspace Method Infrared vs. Visible Face Recognition”, Machine Graphics and Vision, Vol. 18(1), pp. 47-66, 2009.

Conference Papers:

S. Hati and S. Sengupta, “A GA-Based Integrated Approach to Model-Assisted Matching and Pose Estimation for Automated Visual Inspection”, IEEE Congress on Evolutionary Computation, Portland, Oregon, USA, June 20-23, vol. 2, 1346-1353, 2004.

S. Hati and S. Sengupta, “A Genetic Based Approach for Robust Pose Estimator”, Asian Conference on Computer Vision, ACCV-2000, Taipei, Taiwan (Jan. 8-11).

S. Hati and S. Sengupta, “Robust Camera Parameter Estimation using Genetic general Algorithm”, IEEE International Conference on Systems, Man and Cybernetics, IEEE languages International on Systems, Man, and Cybernetics, Tokyo International Forum, Tokyo, Japan (Oct. 12-15, 1999), Vol. 4, 943-947, 1999.

S. Hati and S. Sengupta, “Pose Estimation in AVI using ANN”, IEEE International Conference on Systems, Man and Cybernetics, San Diego, California, USA (Oct. 11-14, 1998) Vol. 2, pp. 1732-1737, 1998.

S. Hati, S. Sengupta and M. Mukherjee, “Generation of Perfect Line Drawing for Automated Visual Inspection”, International Conference on Robotics, Vision and Parallel Processing for Industrial Automated Application (ROVPIA 96), November 28-30, 1996, University of Sains, Ipoh, Malaysia

M. D. Shahbe and S. Hati, “Decision Fusion Based on Voting Scheme for IR and Visible Face Recognition”, Fourth IEEE International Conference on Computer Graphics, Imaging and Visualization, 15-17 August, Bangkok, 2007.

A. W. Reza, C. Eswaran and S. Hati, “Blood Vessel Detection using RGB Components for Early Diagnosis”, Proceedings of International Conference on Robotics, Vision, Information and Signal Processing (ROVISP07), Penang, Malaysia, No. 43, pp. 515-519, November 2007.

C. Eswaran, A. W. Reza and S. Hati, “Detection of Optic Disc in Color Fundus Images Segmented Objects in a Real Time using Green Channel Components”, Proceedings of Second International Conference on Informatics, Malaysia, No. 5, pp. 27-33, November 2007.

I. Abdurrazq, S. Hati and C. Eswaran, “International Conference on Computer and Communication Engineering. 1373-1377, Kuala Lumpur, Malaysia 13-15, May 2008.

M. D. Shabe and S. Hati, “IR and Visible Face Recognition using Fusion of Kernel Based Features”, 19th International Conference on Pattern Recognition (ICPR) Tampa, Florida, USA, Dec. 8-11, pp. 1-4, 2008.

Contact this candidate