Henry Truong

Cell: 408-***-****, Email: adobmq@r.postjobfree.com

OBJECTIVE

Seeking a senior technical and/or managerial position in the areas of Cloud (IAAS, PAAS, SAAS, IoT, Security) Testing/QA/QE, System Performance/Scaling Analysis/Tuning/Optimization/Testing or DevOps/DevSecOps/SRE.

SUMMARY

Hands-on and management of Test/QAE/Development/DevOps/SRE and Architecture projects.

25+ years with 10 years in development and 15+ years in technical/hands-on QA/SDET/DevOps & Management. Broad knowledge and experiences in networking (L2 to L4+) and in the last 10 years in IAAS/SAAS/PAAS cloud platforms, with focus on performance/scaling/resiliency, tuning and analysis.

Have built/mentored/managed teams of 5-50 on-shore/off-shore engineering teams, many are QAEs/SDETs.

Worked at many companies as director, senior manager, architect, and principal engineer.

Define test strategies, processes, criteria and test plans for various test phases (DEV, FT, ST, PRD)

Architected and implemented many test automation frameworks on many projects for both QA/QE & DevOps

Result-oriented, strategic and agile with focus in quality, performance/scaling, resiliency, and security.

Research on cloud deployment of (micro-)services in Kubernetes, AWS EKS with Ansible and Terraform.

EXPERIENCES

LinkedIn (through Harvey Nash), Sunnyvale, CA, 03/2020 – now

DevOps SRE Contractor, GRID SRE Group

●Work on various SRE tools for big data Hadoop GRID cluster management:

oAutomation of the on-premise cluster management (expansion, migration, decommission, auditing of Hadoop clusters of 20K+ nodes across multiple region colos)

Jenkins self service pipeline design and automation (groovy, bash, python)

Extending and developing base service access layer in python to interface with Jenkins self service jobs, Kerberos, JIRA, LDAP, Kafka,Slack, and other LinkedIn DevOps systems.

●managing all the GRID SRE Jenkins-based self services.

●looking into developing SRE tools to put all current Hadoop services (KDC, LDAP, Kafka, Spark, Presto, Azkaban, …) running on the on-premise GRID to K8S Kubernettes and migration into Azure.

NTT Data (Wells Fargo client), Fremont, CA, 04/2020 – 02/2021

Software Development Advisor, DevSecOps SRE Group

Advise and help Wells Fargo DevOps tool group with architecture, implementation, and maintenance of DevOps automated tools and application pipeline jobs and tools:

●Enhance, re-architect and maintain CI/CD Jenkins pipeline tools in python

oAutomated codes talking to Jira, github, Jenkins (some groovy) to run pipeline build & release jobs

oAutomated tools for infrastructure checking, build metrics (could do it in python numpy & panda)

•Lead developer for DevOps node.js/express.js API services for management of configuration properties and application on-board process (involved with react.js UI development, but mainly responsible for all backend API services’ design and implementation).

Enlighted Inc., a Siemens Company, Sunnyvale, CA, 09/2018 – 04/2020

Principal Full-Stack (DevOps SRE & QE) Consultant, Enlighted Technical Operation (DevOps/IT/QE) Group

Responsible for SRE system testing and DevOps automation for Enlighted IoT products and cloud solution:

•Responsible for all frontend API and sensor/gateway backend Cloud SRE

•Lead performance analysis and reliability, scaling testing and tool development on backend cloud services which include clusters of Haproxy, Consul, Redis, Spring app servers, Kafka, Storm, Cassandra services.

oDevelop test automation framework tools and test suites using JMeter (scripting in Java and python):

Automation of SRE testing from different cloud interfaces for different product versions:

•Management/Orchestration interface: simulating user REST interfaces hitting Tomcat application servers, reddis and Cassandra DB from the user side, and

•Sensor & Gateway interface: Sensor data injection from the gateway side through Kafka, Storm, and Cassandra pipeline running on different environments on VM-based GCP cloud.

Automate test reports and tools for TPS, Latency, Memory, integrate them into Jenkins CI

•Test Framework and automated scripts include performance analysis and problem detection tools and reports for LB haproxy/snapt log, JVM GC log, HeapDump, and ThreadDump

Help fixing and trouble-shooting a number of system performance and scaling issues, such as memory, sockets, CPU, threading, IO, load-balancing algorithms for the product big data backend.

•Work on and fix web (Spring Boot) services and data (Kafka/Storm/Cassandra) services’ development codes.

•Responsible for GCP cloud deployment using automated Ansible and bash scripts (Nexus for artifacts)

•Maintain CI/CD Jenkins – help build/configure/automate/run/monitor pipelines, Sonar, Dev, QA, DevOps jobs.

•Help in evaluating/choosing, developing and maintaining other DevOps tools and cloud infrastructures: LBs (haproxy/nginx/snapt, i.e. LB ADC, WAF, Caching, SSL-offloading), Firewall, Splunk.

•Look into security and privacy testing and assessment for the entire IoT product (frontend & backend)

oBackend penetration testing, vulnerability, data leak, data loss, access control, authentication/authorization (SSL/certificates), data storage using tools nmap, OWASP ZAP, SonarQube.

oFrontend Browser/mobile app privacy – autofill, account, fingerprinting

Comcast, Sunnyvale, CA, 08/2017 – 04/2018

Principal Cloud SRE Contractor, Advanced Security Group

Responsible for architecting, integrating, system testing, and (DevOps) laying out of an integrated security product, which involves millions of Comcast CPE Linux-based gateways and security analytics AWS cloud (Linux Ubuntu VMs), doing device fingerprinting, security checking, and parental control:

●DevOps:

oWork with partner company to architect and deploy services to various AWS regions, VPC, subnets (public/protected/private), Availability Zones to map analytics services, i.e. application service, Kafka messaging service, partner API (websocket) services, RDS, fingerprint/security services (user device analytics, url-checker, ipset, speed-test, parental control, etc.)

oResponsible for the implementation of product compliance to Comcast security standards as the modems in the Comcast HFC network are connected to the AWS analytics cloud through a direct shared link.

oAdmin the AWS dev/test staging, scaling test, and production clouds.

oConstantly monitoring application performance metrics with NewRelic and Admin tools

oCost planning and analysis through capacity modeling test (write scaling test in node.js)

oSet up and monitor webapp (mortar) Jenkins branch builds. Use github, gerrit, Jenkins.

oDeploy webapps to Cloud Foundry PAAS environment (PCF) and monitor them.

●Testing:

oHelp automating security product functional test framework and components:

End-to-end test (gateways -> analytics cloud -> API cloud -> gateway UI) in python

oAutomate and run Cloud Scaling tests (up to 15M gateways, 15 devices/gateway) in node.js (REST over WebSocket over STOMP). IDEs used: Visual Studio Code, IntelliJ.

oDo performance analysis, capacity modeling, and trouble-shoot system queueing issues.

oDeveloped UI automation tests using Selenium and Appium frameworks

●Use many monitoring tools such as CloudWatch, Tick/Chronograph, NewRelic, Kibana, and Splunk

●Define and monitor key application and system performance metrics

●Participate and perform security review of the whole product. Familiar with OWASP ZAP (VA/port-scanning (nmap)/Pen-testing/Fuzzing-testing), OAuthv1/v2, Certificates, SSH-CA, Ansible Vault.

AutoDesk, San Francisco, CA, 03/2016 – 08/2017

Principal SDET Contractor, Cloud Licensing and Order Fulfillment Systems

Working on AutoDesk Cloud Licensing and Fulfillment Services product integration/QA/system testing with almost 100% test automation approach (AWS server cloud on Linux Ubuntu EC instances, docker containers running undertow, http-proxy, consul, Dynamo DB, ES, graphite/Datadog, Kibana/logstash, NewRelic)

Licensing Service Cloud:

●Lead QE/Test architecture, automation design and implementation:

Developed UI Functional test suite using Selenium and TestNG frameworks, integrated it into Jenkins Selenium Grid

Maintained QA Functional API test automation in python (pytest framework)

Designed and implemented most parts of the new test automation using node.js using Mocha test framework

Own system testing - implemented JMeter tests for Performance, Scaling, Stability, Stress, Auto-Scaling, and node.js tests for HA/Resiliency. Good understanding of OS/System performance metrics (OS kernel, I/O, TPS, latency, component/cluster input/output queuing).

●Maintain a java8 & spring framework mock service for mocking all DEV and INT REST endpoint services test data to help in parallel TDD test automation development.

●Pioneer all system leveled product specific performance and API endpoint metrics and logging/tracing capabilities (TPS, latencies, count, error rate, requests, responses, etc…) with DevOps and Development on DataDog and Kibana.

Order Fulfillment Service Cloud:

●Lead QE/Test architecture, automation design and mock server development:

Design and implement test automation framework in Java, TestNG, Rest-Assured, etc…

Build a java spring-based mock server for mocking of interfaces around the order fulfillment system, namely backend REST interface, SOAP WS,TIBCO EMS, and SAP IDOC.

iControl Networks, Redwood City, CA, 08/2014 – 03/2016

System Test Architect, Internet of Things Server Cloud Group

Mainly responsible for architecting and managing all test automation and testing of all system testing areas, and all IoT Server Cloud Performance, Scalability, Stress, and Resiliency/HA testing of the company IoT home security & automation cloud which managing & controlling messages and alerts from home/business sensors, live/scheduled cameras, thermostats, outlets, door locks, etc…

Audio/Video (VOD/LiveStreaming) Scaling Test was done with JMeter (m3u8, adaptive bitrate, HLS)

Test Tool Development (Java), System Test Automation (Java, JavaScript, bash), and System Testing (HA Resiliency, Performance, Scaling, Stability, Capacity Modeling/Planning) of:

oBackend cloud service clusters: httpd, haproxy, tomcat, Oracle, XMPP, RabbitMQ, Kafka, Zookeeper.

otesting/automation using open-source test tools, own developed proprietary test tools (Java, bash)

oAutomation with bash, Java, and REST API system scaling/stress with JMeter, some tsung.

oStatistics collection and analysis with Logic Monitor and graphite besides normal OS & JVM tools ps, top, lsof, netstat, vmstat, iostat, sar, free, jvisualvm/jconsole, jps, jstat, jmap, Jenkins, and write own stat tools.

Responsible for internal/external customer deployment/solution troubleshooting & analysis

oOptimizing system performance and scaling. Solved a number of critical internal issues

oPartly helping DevOps to analyze and trouble-shoot customer issues, reproduce and perform customer solution testing, and propose fixes or solutions to DevOps for customer deployment (Linux VM cloud)

Functional Test Automation in Python (pytest framework).

Influent directly and indirectly all QA testing activities, besides system leveled testing activities.

Prototyped and tested clusters of Cassandra and Spark for a short time

Familiar with many IoT devices (Linux-based IoT devices, Hue, LiFX, bands, Ring, Chamberlain, HomeBoy, cams, Sonos, Netgear Alos, etc..)

Nokia, Alcatel Lucent, Nuage Network Division, Mountain View, CA, 08/2012 – 08/2014

Sr. Manager & Architect of SDN Cloud Virtualization QE Test Group

Managed a team of 15 engineers doing testing/automation on development sanity, integration testing for UI, API, Functional and System leveled testing with tight Agile build structure in Jenkins.

Architect/Implement the SDN cloud virtualization VSD PaaS/SaaS QE test infrastructure, automation framework and test releases for testing clusters of JBoss, ejabberd, MySql, Hadoop, HBase, Cappuccino GUI to manage VmWare & OpenStack VM’s (all VMs are Linux CENTOS based)

Backend Functional Feature Regression Testing and automation was done in Python Robot framework.

System Performance/Scaling/Stress/HA test automation are in Java & JMeter.

UI Functional test automation in Ruby and Cuccumber.

Drive/Participate in all semi-SCRUM-like cycles, QE activities, standards, practices and criteria.

Write a number of test driver clients and simulators for Google Chat ejabberd XMPP, REST API, JMS, and HTTP as plugins into JMeter, mostly in Java & JavaScript.

Wrote SDN orchestration server and SDN controller test simulator and test clients using XMPP in Java as a plugin to JMeter, where Functional, Performance & Scaling test cases are driven from.

Work on Ruby, Perl scripts using Cucumber & CucApp for our Cappuccino GUI test automation.

Write bash scripts, python and Tcl/Expect scripts for getting accessing machine infrastructures.

Infrastructures: VMs & Hypervisors on CentOS, RHEL, OpenStack, vCenter/vCloud, CloudStack, etc..

Other Testing: NMAP, snort, vulnerability assessment, LDAP, REST API security/penetration.

Yahoo, Yahoo Mail Cloud, Sunnyvale, CA, 10/2009 – 08/2012

Principal QE Engineer, Yahoo Mail Cloud Backend QA/QE

Test Tool Development & Automation for both Functional and System Leveled Testing,

Performance/Scaling/Load/Stress/Capacity Testing, Analysis of Linux RedHat Mail Cloud Performance.

Test Tool Development / Automation Architecture & Implementation:

●Introduce, architect, and implement QA test tools and test frameworks for unit, api, functional and system leveled testing – design and implement many Java JMeter plugin test clients to automate API/Functional/System (scaling/load/capacity) testing of the Mail services. Received award for test tool development

●Propose/Design/Implement JMeter + TestNG + Selenium2 test framework (JMeter launched parallel Selenium tests).

●Technical Lead for the IMAP/POP Mail Performance/Load/Stress/Scaling Testing & Automation

●Simulation of millions of user thread traffic coming in from mobile phones, desktop clients, and web service, and browsers (selenium-based) – test clients with very high level of concurrency.

●Design and Implement a J2EE Statistics Collection and Data-Mining application server in Groovy and Grails and Java running in Apache Tomcat with MySQL DB.

System Leveled Testing & Analysis:

●Leading component/system/Cloud Performance/Load/Stress and Capacity Planning & Modeling using well known Capacity planning/modeling theories and laws to predict software/hardware utilization for hardware ordering in the cloud and software performance optimization tuning on new cloud deployment.

●Pinpointed a number of performance and system issues where the threads are blocked or deadlocked, race conditions, most time-spent codes, hotspots, exceptions,memory with tools: YourKit, jps, jstat, jmap, jhat, jconsole, visualvm, and other performance tools such as sar, (h)top, mpstat, netstat, and Project R.

Cisco Systems, San Jose, CA, 02/2008 – 12/2009

Technical Lead Contractor, Sports & Entertainment Solution Engineering, AS Group

Received ‘Outstanding Performance’ award while working as a consultant.

Key Responsibilities include test planning, scoping, hands-on Integration/Functional/System test case writing & execution, test automation and training of most QA engineers for 7 SCRUM releases.

Found a majority of bugs in QA, of which were challenging system leveled bugs

End-to-End Solution testing & automation for the Digital Signage Java Media Server running in tomcat and MySQL DB, Flex/Flash GUI, IPTV Flash-based Digital Media Player set-top boxes.

Test Automation & Tool Development: JMeter, Emma, FlexMonkey, JDBC

Worked with Linux RHEL 5.2, MySQL, Digital Media Players busy-box firmware/kernels, js prototype library, scripts, flash browser, VLC client/server, integrated VOIP phones (standard SIP phone), video equipment such as Scientific Atlanta Cisco Analog to Digital IPTV Transcoder, Channel-Mapping DCM, IP-Phone CM & CUAE, DSM, POS, ad insertion tools (Adtec), VideoLAN client/server tools.

QA Lead Contractor, Deep Packet Inspection Application Service Module, AON/PMBU

Tested low-end deep packet inspection appliance with focus on controlling different web traffic protocols, mainly HTTP/XPath/XML/JMS/REST (some WebDAV). The appliance OS is a multi-core, multi-threaded real-time kernel (MV CGE 5.0), JVM 1.5.x and Cisco PCube SML (Cisco DPI).

Helped with management of all QA resources, budgets, lab, projects & testing.

FOREX/Currency Trading, Founder, Partner @UpRiseCapital, San Jose, CA, 01/2006 – 01/2008

UpRise Capital is mainly just a small investment club for retirement and hedge fund piloting purposes.

Implemented the “CashAdvisor” product - a software-based Technical Analysis Auto-Trading system using MetaTrader MetaQuotes (C-like) programming language. CashAdvisor automatically place and exit orders, gives buy/sell signals through emails and alerts, and posts results on web site.

PacketMotion, San Mateo, CA, 11/2004 – 12/2005

Director of QA

Led the company QA group and direct all QA activities of PacketMotion PacketSentry product line, which is a deep-packet inspection and policy enforcement switch.

50% hands-on QA automation work, 50% on process and people management – researching, architecting, test plan writing/reviewing, writing test program and traffic generator development

Testing application security features such as malicious URL strings, buffer overflow in application and data base, URL redirection, phishing, identity spoofing & stealing, illegal file copy/access.

Nokia, Mountain View, CA, 03/2000–09/2004

Director of QAE (3rd Line Manager), Mobile Packet Core @Came from the Nokia acquisition of Amber Networks (Service Edge Router) as Sr. QA Manager

Managed, architected & led directly and indirectly the global (US, India, Hungary, Finland) I&V / QE test automation effort (Integration & Verification, same as QA) organization of around 50 engineers, 6 1st and 2nd line managers and contractors, doing QA Testing (grey/black box testing, test automation).

Architected and implemented the base test automation framework for switch/router testing for both Amber Networks and Nokia Mobile Packet Core 3G Router. Knowledgeable of L2 to L7 switching/routing protocols.

Wrote, reviewed and approved all IV/QA test plans, program plans, and some development specifications.

Led the organization to CMMI Level 3 certification and tools for Quality assessment and achievement

Helped manufacturing group setting up test benches and training/running ATE/diagnostics tests

Founding Consultant @FlowGate, Inc., San Jose, CA, 04/1997 – 03/2000

Hands-on & running 10+ contractors on various development & test projects for clients in networking industry:

1.Consultant @ Com21, Inc., 07/1997- 03/2000

●Managed a small group of full-time/part-time contracting engineers to architect & implement a complete end-to-end test automation system and testing for many Com21 products (DOCSIS 1.1/1.0 and proprietary cable modem/headend with data + VOATM).

2.Consultant Developer @ FlowWise Networks, Inc. 04/1997 - 07/1997

●Architected and implemented a Java-based Device Management Application for a workgroup Gigabit Ethernet L3/L4 Switched Router, using Java over SNMP with VisualCafe, AdventNet toolkit.

●Oversee SNMP agent and MIB development on the switch; SNMP agent written in C; MIBs include MIB-2, If Extensions, Bridge MIB, RIPv2 MIB, RMON, OSPF, FlowWise Proprietary MIB.

Member of Technical Staff @Network Peripherals Inc., CA, 01/95 - 03/97

Defined the Network Management product requirements and led the engineering development effort to:

●Design and implemented multi-OS management Agent/EMS applications for the Ethernet-to-FDDI switch: (1) UNIX OpenLook/XView/MOTIF/C based NMS GUI, (2) Windows Visual C++ NMS GUI.

●Implement the embedded SNMP agent on the switching hub, and designed part of NPI MIB's.

●Integrate NMS products with Sunnet Manager, HP OpenView and SNMPc for both UNIX and PC.

Sr Engineer @ Nortel Networks (Bay Networks, SynOptics Communications)., CA, 01/92 - 01/95

Embedded System development for Ethernet and FDDI switches and concentrators (L2 to L4 services)

Mainly responsible for network management agent development (SNMPv1/v2) in C.

Development Environments: Intel x86, ICE, VxWorks RTOS, and Sun OS/Solaris.

EDUCATION - San Jose State University - MS Computer Engineering (3.6 GPA), 05/1994

RESIDENCY STATUS - U.S. Citizenship

REFERENCES – available upon request

Contact this candidate