Post Job Free

Resume

Sign in

Software Engineer Big Data

Location:
Menlo Park, CA
Posted:
November 27, 2023

Contact this candidate

Resume:

KAYA ALPAN

US Citizen

Palo Alto, CA

808-***-****

ad1hzi@r.postjobfree.com

PROFILE

https://www.linkedin.com/in/kaya-alpan-1691aa3/

Results-driven and proactive problem solver: a versatile professional with expertise as a polyglot Software Engineer, Data Engineer, Architect, DevOps/SRE Engineer, and Technical Leader, characterized by a customer-oriented mindset. Architected and developed large-scale distributed systems with high-availability, high-throughput and big data needs in elastic and self healing cloud environments. Current experience: IoT, Big Data, Kafka, Postgres, Athena, Snowflake, Streaming and Batch ELT and Analytics in AWS, Core Java, Enterprise Java, Spring Framework and other frameworks and languages.

· Self-motivated with natural ability to take a project from conception through delivery.

· Effective execution of broad and deep technical knowledge to create synergistic solutions.

· Technical leader, mentor and team player with excellent communication and Agile development skills.

· Experience with both well established and highly structured environments as well as high-paced start-ups.

· 10+ years production level experience architecting, developing and budgeting for AWS Cloud.

· 20+ years technical leadership and management experience as Lead Engineer/Architect/Consultant/Manager/Director/VP.

· 20+ years end-to-end experience in architecting and developing distributed/enterprise Java applications and APIs often with high-volume as well as complex transactional requirements.

· 15+ years of experience working with data and database driven systems (SQL). 6+ years of experience with Cloud, Big Data, NoSql.

· 10+ years of AWS DevOps/DataOps experience. Data Center to AWS migration experience. Ability to architecture for AWS, mindful of cloud elasticity, virtualization and cost. Developed low-level features for ServiceNow cloud used by ServiceNow SREs. Current experience of IaC with Terraform, Ansible, CDK, CloudFormation.

· At home in *NIX environments, proficient in bash/ruby/python/typescript/HCL scripting.

· Over 10 years of remote work experience. Over 5 years of DoD and Federal (VA) work experience as a contractor.

· Experience with electronic medical records and sensitive data (HIPAA, PHI, PII data).

· Experience with leading cloud and technical aspects of SOC2 compliance efforts. PROFESSIONAL EXPERIENCE – in reverse chronological order Employer: Outset Medical July 2020 – Present

Employment Location: San Jose, CA and Remote (Hybrid) Nov 2022 - Present: Senior Staff Software Architect July 2020 – Nov 2022: Senior Manager & Architect,

Senior Staff Software Architect

Owner of production infrastructure and reliability of services. Provide technical leadership and guidance to the Cloud Infrastructure and Platform team and across teams. Focused on mission critical data work that often involves a standalone Kafka cluster, Relational

(Postgres) and No-SQL databases/data stores (Athena, S3, DynamoDB), cloud native data streaming applications.

Designed, developed, provisioned and deployed mission critical Java, Scala, Python and Typescript data streaming applications and APIs with integrated monitoring to EC2 instances, ECS/Fargate instances and Lambdas. Leveraged consumer/producer pattern with Kafka topics, SQS and Eventbridge, Kinesis data streams to facilitate big data streaming and ELT that powered predictive analytics as well as BI dashboards. Utilized parallelization, concurrency and caching to build robust and performant cloud-native reactive systems with high throughput and low contention. Architected, provisioned and managed cloud infrastructure using Terraform, Ansible, Docker, Jenkins, AWS CDK, CloudFormation. Owned solutions end-to-end. Mentored and led DevSecOps and Platform engineers, application developers. Architected and developed IoT big data streaming solutions.

Led and provided deep expertise in troubleshooting cloud networking, complex distributed application and database issues and outages, 24 x 7 monitoring and support of production resources. Performed routine upgrade of complex systems: Kafka cluster, RDS/Postgres. Performed big data migrations between S3, Snowflake, Postgres. Sole IC of the infrastructure and application replication of our cloud within Veterans Affair Enterprise Cloud.

Cloud Security & SOC2 Compliance - Architected and developed cloud security solutions to address OS and containerized application vulnerabilities continuously: leveraged AWS inspector, Vanta, Tenable and Crowdstrike to detect vulnerabilities; Created AWS SSM patching resources using terraform in order to automate continuous patching of EC2 instances; streamlined Dockerfiles, Java and Scala application libraries (maven pom.xml, and sbt build files) to eliminate vulnerabilities; mentored and led team in this regard to improve cloud the security posture of Tablo Cloud.

Proactively defend Tablo Cloud resources against malicious attacks through proper network ACL and security rule configurations.

Sr Manager & Architect

Juggled roles of Engineering Manager and Architect for the Cloud Infrastructure and Platform team. Owned production infrastructure and all production systems. Owned VA-EC technical deliverables.

Interviewed and hired engineers to build my team. Led, mentored and collaborated with my team members as well as other teams in architecting and implementing DevSecOps and data solutions. Provided weekly updates to upper-management about status of Cloud projects, leveraging Jira for project management and planning.

Architected, designed, developed, maintained and monitored end to end cloud IoT data streaming applications in AWS commercial and AWS GovCloud. Acted as gatekeeper of Outset productions systems.

Developed and managed infrastructure using terraform, ansible, AWS CDK, CloudFormation. Developed infrastructure, data streaming and ELT applications with HCL, Python, Java/Core & Spring Boot, Scala/Akka, Typescript and SQL.

Developed monitoring solutions using AWS CloudWatch with terraform and CDK. Replicated our production cloud environment within a GovCloud account to prepare for the FedRAMP high VA-EC environment.

VA-EC: Acted as Architect, SME and IC in replication of our AWS cloud within VAEC GovCloud (Veterans Affair Enterprise Cloud). Security cleared to operate in a production environment within VA-EC.

SAC cleared to work production workloads via zero account in VA-EC. Created multi-account hierarchy leveraging AWS Control Tower and Organizations in order to implement security and compliance measures across all cloud accounts. Employer: nops.io February 2020 - June 2020

Employment Location: Palo Alto (Remote), CA – VP of Engineering, Chief Architect Core challenge: to bring technical order and sanity to an early stage, pre-series startup blessed with a promising value proposition and ridden with chaotic engineering mal-practices. Led and managed independent software contractors that are distributed across multiple time zones and pertinent software projects, acting both as a hands-on engineer as well as the technical lead of the software team, providing architectural guidance as well as technical direction as the principal technical decision maker and best practices enforcer, while also building and maintaining relationships with nOps’ clients and other nOps employees and contractors among others. Focus: AWS specific cloud change management primarily across cost and security pillars of AWS well architected framework, leveraging Billing, CloudWatch and CloudTrail logs. Tech stack: Java, Scala, Python, Django, React, Elasticsearch, AWS. Employer: Outset Medical June 2018 – February 2020 Employment Location: San Jose, CA – Senior Data Platform Engineer Core responsibility: SME and hands-on technical leader of Outset Cloud and Data platform, providing direction and blueprints for a Cloud Native Distributed Data Platform. Re-architected dysfunctional batch and streaming ELT data pipeline and actively developed:

- Tools for data ingestion:

Kafka: real-time ingestion of time series data generated by IoT (Tablo hemodialysis machines): ca 130+ sensor readings per second per machine. Kafka consumers in Java/Scala that partition and persist ingested data to AWS S3 real-time.

Real-time anomaly detection with Spark Structured Streaming. Athena CTAS queries to perform batch periodic data compaction from json to parquet via Python, AWS Lambda and AWS CloudWatch.

- Daily data QA via Python, AWS Lambda and AWS CloudWatch.

- Productionalize reports, queries for Data Science needs.

- Working in a Jenkins, Ansible, Docker, Terraform and CF managed AWS environment.

- Initiated work to refactor existing Monolithic backend apps into cloud native, reactive microservices to achieve a better separation of concerns and clearer service boundaries.

- Migrated Data Warehouse from Athena/S3 to Snowflake/S3, leveraging S3 event triggers and Snowflake pipes to populate DW from IoT data streaming real-time into S3.

- Architected and developed a data pipeline with Kafka and AWS Elasticsearch to power dashboard charts.

- Architected Finite State Machine driven over-the-air (OTA) software update lifecycle of IoT devices.

- Drive and influence the evolution of our data and cloud infrastructure.

- Review of architecture and code developed by the team.

- Provided upper management with strategic and tactical technological insights. My architecture allowed us to move our focus from daily data QA headaches to value-added goals, such as enabling work toward ML/Predictive Analytics. Worked with AWS S3, AWS Athena, Snowflake, Elasticsearch, Apache Kafka, Apache Spark, RDS (Postgres), AWS Lambda, AWS DynamoDB, CloudWatch, developing mainly in Java, Scala, Python, SQL and Bash scripting.

Owned technical roadmap: cloud native, event-driven data services and streaming pipelines, storage and application ecosystem powering actionable predictive analytics across capacity, availability, performance as well as cost dimensions. Employer: ServiceNow May 2016 – May 2018

Employment Location: San Diego, CA – Staff Software Engineer Developed:

- Cassandra and MySQL engines for time series proof-of-concept work.

- - Benchmarking framework and CLI tools for time series database.

- - Real-time data ingestion client that consumes roughly 100K * 125 floating point metrics per minute from production Kafka cluster and persists into a time series database.

- - Features for provisioning infrastructure of MySQL and Java servers in ServiceNow platform (Ruby, Bash, Puppet, SELinux, Vagrant).

- - Core SN platform development (Java, Javascript/Rhino). Work included design, development and troubleshooting of mission-critical features and components.

Employer: Fidelis Education Oct 2013 – May 2016

Employment Location: Palo Alto, CA – Lead Architect/Developer/DevOps Engineer (last title:Director of Engineering)

Lead Software Architect, Developer & DevOps Engineer in an AWS cloud based multi tenant SaaS environment at early stage startup.

Helped migrate Fidelis to a fully automated and elastic Cloud Ops powered by AWS OpsWorks/CloudFormation.

Developed full-stack application features in SaaS application: Spring Framework, Backbone, MySQL RDS, DynamoDB, Solr, Elasticache.

Designed and developed microservices/APIs in Java, Scala, Javascript, Ruby:

- Multi-tenant SaaS security infrastructure that enables SAML2 based SSO.

- User CRUD API in Scala with a MySQL RDS and DynamoDB backend using Play Framework and Akka.

- Dynamic SQL script and report generator tool set in Javascript/NodeJs

- Customer configuration data API against AWS DynamoDB using Scala/Play/Akka.

- Activities API using Scala/Play/Akka and AWS Kinesis.

- HAProxy for API redirects: created configuration and Chef cookbook to be used by OpsWorks.

- AWS Lambda written in Scala that sanitizes large compressed and non-compressed JSON files.

- Data integration/ETL pipeline powered by Apache Spark and AWS Elastic Map Reduce service as well as the APIs for scalable real-time and batch data consumption. Provided 24x7 cloud network security and monitoring. Initiated and led implementation of AWS OpsWorks/Chef, Github and Atlassian Bamboo driven CI/CD pipeline.

Learned on the fly to implement secure cloud solutions with AWS VPC with public and private subnets, NAT, Bastion, IGW, VGW, ACLs, Elastic Load Balancers as SSL endpoints, among other AWS services. Created CF templates, a multitude of Chef cookbooks and recipes to manage OpsWorks stacks, their layers and server instances therein. Spearheaded Scrum for development and Kanban for DevOps throughout the organization. Employer: Playfirst Inc May 2012 – Oct 2013

Employment Location: San Francisco, CA – Lead Architect/Developer/DevOps Engineer (last title:Director of Engineering)

Lead Software Architect, Developer & DevOps Engineer at a mobile game company with over 20 million users.

Game APIs

Designed and developed Game Play API in Java to store real-time game play state data. Designed and developed Game Event Data Collector API to collect events from mobile devices in real-time and its big data analytics backend to harness user behavior insights. Refactored Advertising API from Rails to Java/Spring. Performance tuned, load and stress tested APIs under massive loads to simulate multi-million user spikes.

Big Data/NoSql: Analytics and Reporting

Architected and developed a solution that's based on Hadoop (CD4 and AWS EMR) cluster and Redshift used for daily analytics data processing and reporting. Cloudera Hadoop cluster: developed multitude of Hive jobs, Map Reduce jobs, Redshift scripts, Sqoop jobs and automated via Oozie workflows.

Created interactive as well as script driven Elastic Map Reduce clusters up to 1000+ nodes to process multi-terabyte datasets for analytics and reporting purposes. Worked extensively with Hadoop, Hive, Pig and MongoDB to work with JSON data. Developed Java UDFs for Hadoop.

Web Apps

Created and led creation of numerous Spring Framework based web apps/APIs with RDBMS, MongoDB and DynamoDB backends and ElastiCache (memcached based) for caching. Developed in Java, Ruby, Python, JavaScript, Scala, Pig, Hive, SQL. DevOps

Led backend systems' migration from the data centers to AWS. Implemented HA via latency driven cross-regional redundancy and regional availability zones, leveraging Route53 and ELB capabilities.

Configured API servers for elastic autoscaling designed to handle spikes; put in place health checks, monitoring and auto-recovery features along with alerting via Cloud Watch. Worked with AWS solution architects to warm up server clusters to handle sudden spikes experienced during game promotions. Provided 24x7 monitoring using AWS CloudWatch, PagerDuty among other tools.

Managed and mentored team members.

Employer: macys.com & bloomingdales.com Aug 2011 – May 2012 Employment Location: San Francisco, CA – Technical Lead Lead design and development of server-side lockout and security question & answer for macys.com and bloomingdales.com

Led an onsite and offshore team of 15+ (UI and backend developers, QA engineers, systems analyst) in an agile manner and successfully implemented critical user profile features across macys.com and bloomingdales.com legacy and microservices driven new generation sites: Server-side lockout and Security Q&A.

Both features were successfully released to customer-facing macys.com and bloomingdales.com e-commerce sites that received approximately 10 million hits/day and generated over $1.5 billion in 2011.

Employer: Akimeka LLC Sep 2008 – Jun 2009 and Oct 2010 – Aug 2011 Employment Location: Kihei (Maui), HI and Honolulu, HI Senior Software Engineer – with Active Secret Clearance Full Stack Java Engineer in a Medical Situational Awareness system for our troops deployed in hot zones, that required US Citizenship and active Secret Clearance. Detailed summary of assignments:

Project: TMDS (Theater Medical Data Store)/JMeWs (Joint Medical Workstation)

● Lead developer in end-to-end (UI, middle-tier, database) OLTP side of theater medical data store and DSS that is used by US troops in active theaters around the globe. Project: MSAT (Medical Situational Awareness in the Theater) Prototype Project.

● Developed various features of MSAT portal/mashup application with emphasis on SOA.

● Prototyped real-time web capabilities of asynchronous server push to support dynamic mash-up and 3-D map display of real-time location data.

● Designed and developed several autonomous state machine components to perform scheduled data acquisition from 3rd party providers via Web Services, email and FTP. This project required US Citizenship and active Secret Clearance. Employer: Referentia Systems Incorporated, Jun 2009- Apr 2010 Employment Location: Honolulu, HI

Lead Software Engineer/Architect

Developed parts of a time series database based on the HDF5 file system for real time analysis of hierarchical sensor data.

Led architecture and development of web portal for LiveAction network topology application. Detailed summary of assignments:

Jun 2009 – Oct 2009

Project: T-Rex - real time analysis of hierarchical sensor data stored in an HDF5 file system.

● Big Data: Imported massive amounts of sensor data from Oracle into the HDF5 file system. Created API to query HDF5 data from Java: JMS/ActiveMQ, web services

(CXF/Tomcat), and from SqlServer via SQL, C# CLR procedures, JMS/ActiveMQ, Protocol Buffers.

● Prototyped bidirectional polyglot messaging via ActiveMQ with Java, C# and Python with JSON and Protobuf.

Nov 2009 – Apr 2010

Project: LiveAction dashboard/web portal

● Led architecture and development of web portal for LiveAction network topology desktop application with Liferay Portal server, GWT, Spring Framework and Postgres.

● Prototyped XMPP based interactive real-time information dissemination with GWT, Liferay portal server and Openfire XMPP server.

● Prototyped charting of real-time and static data with Flex/ActionScript, GWT, Java and JavaScript based charting frameworks. Prototyped interactive charting with jPlot, GWT-Chronoscope, Google Visualization API, Google Charting API, Gchart, and more. Prototyped BI reporting and charting with Pentaho.

● Developed web2 application with real-time charting using GWT, Smart GWT, Portlets, Liferay portal server bundled with Tomcat, Kavacharts, LiveAction, Quartz Scheduling.

● Performed agile project management through the role of Scrum Master. Employer: Cloud Creek Systems Dec 2003 – Sep 2008 and May 2010 – Sep 2010 Employment Location: Honolulu, HI and Agoura Hills, CA (brief occasions) Senior Consultant – Java/J2EE Lead Developer

Mostly remote and occasionally onsite work (Except first 6 months): Worked as Lead Full Stack developer for a multitude of projects at startups as well as well established customers: homeandroad.com, ellisontechnologies.com, westernassociates.com, eharmony.com, stickam.com, 3nonline.com

Made significant engineering contributions, over 5 years, to customer facing site 3nonline.com

(www.everbridge.com), which provided a configurable, omni-channel web driven telecommunications messaging platform to be used by corporations, educational as well as government institutions.

Detailed summary of assignments:

Client: HomeandRoad.com

Architected and led development of Ajax driven Web 2.0 site for a start-up in the realm of location aware, configurable search services and recommendation engine. Developed website with jQuery, GWT 2.0, Struts 2 (JSP), Sitemesh, Oracle ADF Faces (JSF), EJB3, SQL, Oracle database and WebLogic, Google Maps API v3, Localeze data. Performed R&D on Oracle Coherence in-memory data grid in conjunction with generating Tag Clouds and prototyped findings.

Client:Electric Cloud (electriccloud.com), Duration: one month. Created Visual Studio Extension, i.e. add-in for Visual Studio 2008 and 2005 to integrate Electric Commander with Perforce, Svn or ClearCase. Developed in C#.

Client: Ellison Technologies (ellisontechnologies.com), Duration: seven months, during 2007-2008. Performed Oracle 10g data migration and integration work with pertinent reporting and various DBA tasks: sqlldr, SQL, PL/SQL.

Client: Western Associates (westernassociates.com), Duration: occasional, during 2008. Performed advanced Java Hibernate and pertinent Oracle SQL tuning and troubleshooting in production environment.

Client: www.stickam.com, Duration: one month, 2007. Performed advanced Java, Jboss 4, Hibernate 3 and Oracle 10g performance tuning and troubleshooting in video social networking production environment.

Client: www.eharmony.com, Duration: one month, 2007. Supported eharmony.com efforts to move their SQL Server 2000 databases to Oracle 10g r2 databases. Client: Custody Calculations, Duration: on and off, one and a half year, 2006-2008. Developed database driven website to provide child custody calculations using: Spring Framework, Spring MVC, Acegi Security, Hibernate, JSP, JSTL, JSP Tag Libraries, JavaScript, CSS, SQL, PL/SQL, JDeveloper, Oracle 10g Database, Oracle 10g Application Server. Client: 3nonline.com (rebranded as www.everbridge.com), Duration: fours years, 2004-2008, Consultant/Lead Developer

Designed and developed critical features of Struts and Struts2 based SaaS Java web application and its sub-applications using J2EE (JSP, Servlets) /Java, BC4J, OC4J, JavaScript, XML, CSS, JSTL, Hibernate, Quartz, ESRI (ArcSDE, Arc IMS), SQL and PL/SQL. This application provided an easily configurable, omni-channel web driven telecommunications messaging platform to be used by corporations, educational as well as government institutions. Designed and developed a simplified version of 3nonline web application for Blackberry devices; Integrated Struts web site with Struts2 web site.

Led localization and internationalization – I18N, L10N work. Led technical integration with a third party framework geared toward high-volume processing of diverse channels, e.g. email (POP3, SMTP), IM, SMS, Pagers, etc under one roof. Designed and developed a job scheduling sub-system in Java using open source Quartz framework as a replacement for Oracle Job Scheduler – DBMS_JOB. Led technical integration with a third party Geo-Mapping system to support 911 messaging capabilities. Designed and developed GIS features using ArcSDE and ArcIMS. Devised and developed pattern based address standardization and used it to geocode feeder subsystems. Enhanced system by adding the ability to retrieve user data by drawing on an arbitrarily shaped polygon overlay.

Designed and developed OCI based JDBC database connection pooling that supports Oracle’s TAF with RAC; Implemented Web Services Security in a clustered Web Cache environment: SSL-based, WS-I based (XML Digital Signatures, XML Encryption); Participated in system-wide performance analysis and tuning efforts.

Freelance: Spirent, Infrastructure Team, Oct 2003 – Dec 2003 Employment Location: Calabasas, CA

Senior Software Engineer/Contractor

Lead Java engineer on persistence layer for Spirent’s next generation Performance Analysis and Assurance product suite. Performed JBoss performance tuning to bring down startup time. Employer: Iris Wireless Development Llc, Nov 2002- Oct 2003 Employment location: Honolulu, HI

Senior Software Engineer / Lead Engineer

Led design and development in Java of multithreaded real-time high volume inter-carrier Short Messaging Service (SMS) Gateways, HTTP and SMTP Gateways to facilitate SMS communication between wireless carriers. Developed JMX MBeans to manage and monitor systems.

Designed and prototyped TCP/IP based real-time SMS message translation capabilities. Architected real-time and pluggable SMS and MMS messaging infrastructure. Designed and developed SaaS Interface for MMS Transcoding web services. Developed MM7 web services support for MMS systems.

Employer: ISDI, Dec 1999- Nov 2002

Employment Location: Honolulu, HI, Senior System Analyst / Programmer Project: State of Hawaii Criminal Justice Data System. Architect and Lead Developer of the new Criminal Justice Data System. Led efforts to design and develop backend and APIs of Hawaii State Criminal Justice system to facilitate 24x7 information exchange between state law enforcement agencies, in order to migrate a 25 year old COBOL system into Java EE and DB2.

Architected and developed an XML driven workflow and rules engine to capture and maintain a mission-critical flow of information.

Implemented complex search algorithms in Java and SQL that included RLM. Architected, developed, tested and debugged a large base of complex Java and SQL code including most of the transactional and non-transactional EJBs. Led efforts in testing and performance tuning of J2EE components, Java code, SQL statements, application server configurations.

Employer: Rare & Dear, 1998-1999

Employment Location: Eleele (Kauai), HI and Honolulu, HI Oracle Database Developer/Consultant, Jan 1998 - Nov 1999 Mostly onsite and occasional offsite work.

June-Nov 1999:

Client: AllAdvantage.com. Developed many of the Oracle8i back-end processes of a dotcom venture using SQL and PL/SQL.

Feb 1998-Oct 1999:

Client: Bank of Hawaii, Data Warehousing. Developed complex reports for Decision Support purposes in Banking Data Warehouse environment,. Performed analysis, design and SQL, PL/SQL development with emphasis on high-performance SQL and PL/SQL turning. Report work included programming of very complex Cross-Sell and Combination of Services for Marketing on a large amount of data.

SQL and PL/SQL development work at National Marine Fisheries, Cheap Tickets and Bay Networks.

Employer: LHS AG, 1997, Zurich, Switzerland, Duration: three months Senior Software Engineer/Contractor

Project: Debugging and programming code segments in million plus line BSCS, a GSM wireless billing system, using Pro*C, C, SQL and PL/SQL on UNIX platform at ISO 9001 certified German telecom solutions firm.

Employer: Global Securities AS, 1996-1997, Istanbul, Turkey Senior Analyst/Programmer

Designed and developed Fixed Income Securities module of a Financial Trading System in Delphi, C++, SQL and PL/SQL.

EDUCATION - reverse chronological order

· Computer Science / Informatics (advanced coursework), Lund University, Lund, Sweden

· Master of Business Administration, Major: Finance and Banking, C.W. Post Campus of Long Island University, Brookville, New York, USA

· Bachelor of Science – Mechanical Engineering, Major: Applied Systems and Control Engineering, Bogazici (Bosphorus) University, Istanbul, Turkey Passed the National Test of Swedish Proficiency for college level studies. Fluent in English, Turkish and Swedish.

Reasonable comprehension of French, Danish and Norwegian. Learning Spanish.

US Citizen

Clearance:

Secret Clearance (DoD/inactive)

SAC (VA/active)



Contact this candidate