Project Manager System
Resume:
Eric Bullen
Santa Clara, Ca *****
abqbg0@r.postjobfree.com
http://www.thedeepsky.com
OBJECTIVE:
To further use my enthusiasm, dedication, and extensive skills in a
leading and competitive Internet Technology environment.
EDUCATION/TRAINING:
* Partial BS, Computer Science, San Jose State University.
* Veritas Volume Manager 3.1.1 to 3.5 for Solaris
* PMBOK Project Management Level 1 course
* Veritas Cluster Server 1.1.2 (Unix)
* EMC ControlCenter StorageScope
* Solaris 7 System Administration I - SA 237
* Solaris 7 System Administration II - SA 287
* Solaris TCP/IP Network Administration - SA 387
* Platform LSF v. 4.2 clustering software for Linux
* Lion SRS v. 6.0.7 bioinformatics data integration platform
* Network Appliance Data ONTAP System Administration (formerly
"NetApp 101")
* Network Appliance Data ONTAP Advanced System Administration and
Troubleshooting (formerly "NetApp 202")
TECHNICAL EXPERIENCE:
* Internetworking Equipment: Cisco CSS series (formerly
Arrowpoint), HP, 3Com, Foundry Fastiron/Serveriron
* Network Operating Systems: Sun Solaris 2.5.1 to Solaris 10,
Solaris Express CE, Linux (Redhat), FreeBSD, OpenBSD, NetBSD
* Network/System Monitoring: Nagios, Netsaint, MRTG, RRDTool,
snort, symon, ORCA/RICHPse, BMC Patrol, DTrace, truss/strace
* Network Management: UCD SNMP
* Sun Hardware: Sun Fire V880, Netra t 1125, E450, E220R, E420R,
E250, Ultra 1, Ultra 2, Ultra 10, Ultra 60, StorEdge L1000
* IBM Hardware: Bladecenter HS20, X345, and OpenPOWER 720
servers.
* System Management: UCD SNMP, RCS, Subversion, CVS, RedHat,
kickstart, CFEngine, zoneadm/zonecfg
* Network Testing: snoop, tcpdump, Ethereal, Nessus, NMAP
* Lan Topologies: Ethernet, Fast Ethernet, Gigabit Ethernet
(fiber and copper), VLANs, Wireless Ethernet (IEEE 802.11)
* Programming Languages: C, shell (sh, ksh, bash), Perl, Python,
php4/php5
* Storage: Software raid (VxFS, VxVM, LVM, VINUM, Raidframe, Sun
ZFS), Network Appliance cluster, Network Appliance 7xx, 8xx and
9xx series filers, StorEdge D1000 and A1000, IBM Shark 2105,
DS4500, DS6800, and FAStT SAN storage
* Network Services: Sendmail, Exim, Postfix, Procmail,
Spamassassin, Bind 4, 8 and 9, Stronghold 3.x and 4.x, Apache
1.3.x and 2.x, ISC DHCP 2.x and 3.x, OpenSSH, courier-imap,
CUPS, Samba
* Security: GnuPG, Solaris RBAC, PGP, mod_securid (Apache),
OpenSSL (key/certificate generation, etc.), OpenSSH, Cisco PIX,
SOCKS5 proxyservers (Squid), IPCHAINS, IP Filter, PF (from
OpenBSD 3.0+), SecurID/ACE, Tripwire, Systrace, Crack, IPTables
JOB HISTORY:
SR. SERVICE ENGINEER, - July 2007 - Present
Yahoo!, Sunnyvale, California. Responsibilities Included:
* Responsible for one of the top destinations at Yahoo! (Yahoo!
OMG). This includes all aspects of the site, and meeting
existing SLA standards for both site performance and
availability. At peak, omg.yahoo.com would get in excess of 7k
requests a second, which would put significant strain on the
database back end, the API, and multiple caching layers.
* Responsible as the primary engineer for numerous second-tier
properties such as Yahoo! Starwars, Yahoo! Entertainment, and
Yahoo! Shine. This included working with development engineers
and managers to analyze site performance before major releases,
and for troubleshooting issues related to SLA misses.
* Launched Yahoo! Starwars (Worlds)- This included all aspects of
project management, including obtaining hardware by
justification of site performance expectations, as well as
working with many groups (site ops, net ops, etc.) to ensure
goals are met based on established project plan. Later, worked
with the transition of the site's maintenance to our
counterparts in Bangalore India
SR. UNIX ADMINISTRATOR, - August 2006 - July 2007
Shutterfly.com, Redwood City, California. Responsibilities
Included:
* Was responsible for doing a detailed analysis and reporting on
the feasibility of implementing a massive Squid caching layer
in front of the entire storage farm to improve performance, and
reduce load on the back-end. Work included doing a size
distribution of 10's of millions of cachable objects, as well
as their request frequency. After this, determining number of
required systems to achieve a significant level of performance,
and creating a roll out plan for bringing it live.
* Implemented a multi-tiered Nagios installation using a
completely custom roles based administration tool (I wrote it
in Perl) to manage 2.5k nodes, and 25k active service checks.
The design allowed for dynamic slave allocation based on
increased usage, and was responsible for finding limits with
Nagios 2.6's design, and worked with Ethan (Nagios creator) to
improve performance.
* Was the primary administrator for their 4 PB multi-tiered "HSM"
NAS storage environment that was receiving 2.5TB of data a day.
Work included debugging file corruption issues, and NIS issues
with mounting the filesystems.
SR. UNIX ADMINISTRATOR, - August 2005 - July 2006
Netflix, Los Gatos, California. Responsibilities Included:
* Leading a two person team to test IBM's SVC (SAN Volume
Controller) for possible integration into our existing SAN
environment to provide a layer of abstraction for over 200TB of
storage. The SVC project will also include using Veritas' VVR
(Veritas Volume Replicator) to allow for replication of data to
other datacenters, and for DR planning.
* A part of a small team that is responsible for the daily
maintenance of all production servers (mostly all UNIX). This
includes the website, database backend, SAN environment, and
systems to support the nationwide distribution hubs. Through
performance monitoring, analysis, troubleshooting, and
mitigation, we obtain a very high level of performance and
reliability.
* Improved Netflix's kickstart solution with a PHP-based solution
(which I wrote) that allows for very rapid deployment of Linux
servers. The new system allows the delegation of system
building to less skilled admins, and improved the build time
from 1 hour to about 8 minutes.
* Did basic SAN storage allocation for the expansion of database
filesystems. This included assigning LUNs to the system, and
expanding filesystem using VxFS and VxVM. Storage allocation
was done using both the command line and the ESS GUI for IBM
Shark and FAStT storage.
* A part of the migration team to move from the current
bladecenter deployment to virtual servers leveraging IBM
OpenPOWER P720's and Linux.
SR. UNIX ADMINISTRATOR, - February 2004 - August 2005
Nuance Communications, Menlo Park, California. Responsibilities
Included:
* Responsible for the 24/7 upkeep of the entire Solaris/Linux
production environment. This includes all DMZ hosts (web
servers, smtp gateways, etc.) as well as the lab farm, and
internal core corporate infrastructure Unix systems. Total
systems under my responsibility were greater than 200.
* Was the architect and developer of a Perl based inline hybrid
HSM application that abstracts multiple storage devices (in
this case, Network Appliances), and provides version control
for tens of terabytes of data. Features included:
* Capable of directory-level locking during
check-out/check-in. This allows other operations to be
performed in parallel that do not threaten data/file
integrity.
* Wrote the interface as a Layer-4 protocol so that it can
easily communicate with the numerous applications already
in house.
* User account management, and access restriction.
* Ability to merge multiple Network Appliances into a single
virtual directory to provide a uniform file structure to
the users.
* User aware in that during file check-in arbitration, the
user is able to see who previously checked out/in a file
to help determine if their changes can be overridden, or
merged back into the repository safely.
* Was the architect and developer of a corporate-wide spam
solution where none existed prior. I used a Markovian based
spam algorithm as well as custom Perl scripts that operated on
the mail gateway to munge the spam message as an attachment to
prevent malicious code from running during preview. All the
original headers are left intact so the user can still filter
the message as they wish. Learning is all email based- sending
corrections back to the mail gateway so it can learn from its
errors.
* Responsible for the identifying of high-depreciation Sun
systems, and building a migration plan to move those systems to
cheaper commodity based hardware and Linux.
* Due to a severely reduced IT staff, constantly finding ways to
simplify complex systems to help us in managing those
environments. Areas included how Nuance publishes internal DNS
records, user account management, LDAP maintenance, etc.
* Installed a PHP based Linux kickstart solution that allows for
the building of profiles based on IP address- this system
allows for rapid installation of new Linux systems simply by
defining profiles/macros for those systems. I wrote this
application prior to Nuance, and can be found at:
http://freshmeat.net/projects/phpkickstarter/
SR. UNIX ADMINISTRATOR, - February 2001 - February 2004
Incyte Genomics, Palo Alto, California. Responsibilities Included:
* Responsible for the 24/7 upkeep of 60+ production co-located
servers, and 100's of production in-house servers. SLA
agreements required that the production servers have at most 30
minutes a month of downtime for maintenance (99.932% uptime)-
this goal was easily met.
* Was the architect and project manager for the trans-data center
relocation of our two external websites and all equipment.
Responsible for planning the relocation and outage of the
entire lifecycle from scoping, to development and testing, to
completion. This project had extremely high visibility amongst
senior management and was the highest project to be completed
by the company. Accomplishments included:
* Achieved an "outperform" rating from the Director of IT by
bringing everything online in less than 24 hours- the
target goal was 48 hours.
* Was the coordinator between different departments
including networking, facilities, and application delivery
to ensure that project deadlines were met, and
dependencies were resolved.
* Planned the data center floor plan layout- considered
power/HVAC requirements to ensure reliable service, and
efficient rack layout (given limited space) to allow
maximum growth to reduce the risk of moving equipment in
the future.
* Achieved no downtime (some decreased functionality though)
by forcing a split brain in the HA environment,
applications affected included Apache, Oracle, and
Weblogic.
* Was the project manager and architect for the building out of a
network and it's systems for the support of Incyte's new
products. This required project planning spanning multiple
departments (engineering, marketing, and sales) to determine a
best-fit solution by keeping costs down and allowing for growth
based upon expected growth forecasts. Responsibilities included
deciding what platform and hardware to use, user management,
and security risk mitigation.
* Was the architect and builder of the backup system for one data
center to allow it to handle 100's of Gigabytes with a short
backup schedule. Due to strict time constraints, numerous
techniques were required to reduce the window of vulnerability
during upgrades to the data being provided for the online
customers.
* Was the architect and builder of their host monitoring system
to provide an efficient system to monitor 2000+ nodes in
multiple networks. This included writing custom Perl modules to
extend functionality, and creating packages to aid in
duplication of the custom configuration.
* Was the architect for a centralized email processing system to
improve efficiency of analyzing emails sent by the 2000+
production systems. Due to the variety of emails received, this
helped reduce overlooking key important notifications, and
reduced the amount of time spent perusing the emails
significantly.
* Was the architect and builder of a RedHat Linux kickstart
server to aid in building new Linux cluster nodes. I leveraged
the PXE standard, and dynamic DNS (found in Bind 9, and ISC
DHCP 3.x) to allow systems to be rebuilt just from the network
card (not needing a CD or a floppy). This allowed for rapid
build out of new machines in anticipation of increased cluster
processing needs.
* Was the architect and builder of a Linux cluster using Platform
LSF to assist in processing/analyzing multi-terabyte data
volumes to be used for the online product.
* Was a key member in a small security taskforce setup to assess
and mitigate security risks for a new product expected to
generate roughly half of the company's revenue. I am
responsible for the online version of the product.
* Was the sole architect and systems administrator for the entire
web-based presence including the corporate site and
subscription site spanning three data centers. The subscription
site that houses all of the web-based products is used heavily
by many tier-1 pharmaceutical companies. Due to the high
requirements of the customers, security, accountability, and
QOS requirements are extremely high. The systems included
OpenBSD, Solaris 2.6 to 8, and Linux for clustering needs. Due
to the large number of systems supported, a high amount of
automation was required to provide first-class service.
* Supported a group of developers that worked in the above
environments, and provided a high level of service for them.
During my tenure, I received two awards for my services to the
company and their group.
SR. UNIX ADMINISTRATOR, - July 1999 - December 2000.
Riffage, Palo Alto, California. Responsibilities included:
* Responsible for the 24/7 upkeep of the entire
Solaris/Linux/OpenBSD production environment. This includes all
DMZ hosts (web servers, smtp gateways, bastion hosts, etc.) as
well as internal core corporate infrastructure Unix systems.
During my tenure, 40+ systems were under my responsibility.
* During the potential sale of Riffage, a due diligence crew came
and evaluated the production and corporate infrastructure that
I designed and built. The president of Riffage told me that the
crew rated my design as one of the top of about 20 companies
that they reviewed over a two-month period.
* Was the architect and developer of a fully automated file
processing system that was capable of operating over a LAN or
WAN, and also designed to be clusterable across heterogeneous
Unix machines. This program operated continuously for over a
year until a newer version replaced it (which I also wrote).
* Was the project manager tasked to find an adequate data center
co-location, including the management of all contracts and
logistics. Was the systems architect that designed the network
infrastructure, rack layout, and was the architect that
determined specifications and forecasting what hardware would
be required given current expectations. Evaluated countless
vendors demonstrating cutting-edge technologies to determine a
best-fit scenario for the company.
* Was the systems architect that designed and implemented the
corporate network and server environment (which had greater
than 50 nodes and 15 servers), which included OpenBSD, Linux,
and Solaris 2.6 to 8.
* Designed and implemented the corporate mail system, and
production Bind DNS servers (including satellite offices). The
mail system included a specialized mail gateway that filtered
both incoming and outgoing mail using a custom PCRE based
filter essentially making the company immune to popular mail
viruses such as the "Ilove you" virus and variants.
* Designed an extensive auditing system using Tripwire, ingress
and egress firewall rules, and snort reporting that aided in
the identification of numerous attacks and compromised systems,
which brought a swift response.
* Was the systems architect that designed the security policies
that controlled access in and out of the facility. This
includes firewall design, access controls, and implementation
of operating system policies to ensure a highly secure
environment.
* Designed a cost-effective way to obtain high-availability for
the production environment leveraging custom scripts and taking
advantage of inexpensive technologies to reduce costs.
* Created and maintained corporate and production policies.
Directly managed three subordinates. Area of authority included
the multi-million dollar data center (that I worked on below),
as well as the network for central HQ, and offices nationwide.
* Improved and maintained an already existing heterogeneous Unix
environment for a pre-production website- including security
risk assessment and mitigation, creation of a reliable system,
etc. Variants of Unix included Linux, OpenBSD, FreeBSD, and
Solaris 2.6, 7, and 8.
* Wrote a program called mp3_check that is playing a pivotal role
in the quality of the music files available of the website.
After being made publicly available, there have been over 8,000
downloads of the program, and I frequently get kudos from
people around the world. The page for the program is located at
freshmeat.net. People use it from satellite transmission
processing, to extraction of MP3s from Shockwave Flash files.
UNIX ADMINISTRATOR / INTERNET DEVELOPER, Jan 1996 - July 1999.
Diamond Multimedia, San Jose, California. Responsibilities
included:
* Was the architect and developed all of Diamond Multimedia's
Online Store, including the customer interface, database
schema, transaction system, and the administration back-end.
* Maintaining enterprise databases (Microsoft SQL Server, and
Oracle), supporting developers in a Unix development
environment under Solaris 2.5.1 and 2.6, and developing
applications for online/Internet related purposes.
* Created an enterprise level WAN based e-commerce data exchange
system that provided a critical function in managing shipments
of huge quantities of Rio MP3 Player product to third-party
distribution centers nationwide.
* Implemented a remote server administration system for data
replication and clustering capabilities to Diamond's
co-location at Digital Island in Hawaii for their international
customers.
* Created an online HTML page generation system used to build
static pages for certain high-load areas of the production
website.
* Created an HTML based mass emailing system (subscription based)
used by Diamond Multimedia for direct marketing.
* Created intranet-based reporting tools for measuring e-commerce
and direct marketing results of mailings and promotional
specials on the online store.
* Created an extranet for field engineers, allowing secure access
to current product information, including features such as
automatic email notifications of changes and threaded
discussion forums.
* Developed an FTP/HTTP staging server for web development,
including a tiered security model, automatic notification of
files checked-in for quality-assurance, automated
data-replication of approved files to production servers.
UNIX DESKTOP SUPPORT, Aug 1993 to Jan 1996
Diamond Multimedia, San Jose, California. Responsibilities
included:
* Supported hardware engineers and software developers in a mixed
Unix environment. Daily responsibilities included creation of
user accounts, permissions, and troubleshooting typical desktop
problems such as connectivity and application/OS issues.
UNITED STATES ARMY, posted in Europe, August 1989-August 1992.
PERSONAL:
Habits/Interests: Tai-Chi, volunteering for California School of
the Deaf and San Jose Unified School District, running, snow
skiing, gardening, reading, biking, scuba diving, Unix hacking /
programming, world travel, and zymurgy.
Myers-Briggs Type: ENTJ
REFERENCES:
Available upon request.
$Id: resume.txt,v 1.5 2008/09/04 11:13:32 ericb Exp $
Contact this candidate