Senior-level system administrator with over 15 years of experience
in managing heterogeneous UNIX networks, and a strong background in
centralized management of large numbers of machines in a devops style.
August 2011 - Present
Senior Systems Integration Engineer
Part of a team developing Nike's next-generation hosting environment for
all of Nike's production, customer facing websites.
- Used puppet, rundeck, and other tools to manage large, high volume web environment with Akamai, Apache, and JBoss components.
- Designed and deployed Graphite/statsd based metrics collection environment to collect, correlate, display, monitor and alert on thousands of disparate metrics.
- Migrated services from high-cost managed hosting environment to self-managed Amazon AWS environemnt. Developed tools to be hosting provider agnostic.
- Integrated Elastic Beanstalk and other Amazon cloud services into our deploy processes.
- Wrote on-call notification and escalation system using ruby/sinatra, Twilio, and PagerDuty
July 2005 - August 2011
Sr. UNIX Systems Architect
Xerox Office Group, Color Printer Engineering
Languages: Primarily perl and ruby, some python, java, and C.
- Supported mechanical, electrical, and software engineering
- Commissioned and co-designed custom OpenSolaris-based
storage systems to reduce NAS costs by 60-80%.
- Designed and wrote a disk-to-disk-to-tape backup system
to backup 50TB fileservers in a 4 hour nightly window.
- Managed global network of Solaris (10, OpenSolaris, and
OpenIndiana) and Linux (Fedora, SuSE, RedHat, Centos)
machines to support Xerox's engineering organization.
- Wrote APIs to infrastructure and systems administration tasks
to allow lower level sysadmins to more easily help out and business
customers to help themselves when possible.
- Built Linux and Windows virtualization infrastructure
using KVM, Xen, and VMWare ESX. Used Eucalypus API to form
systems into a scriptable private cloud.
- Architected Linux and Solaris web cluster, containing Apache,
Tomcat, and Ruby/RAILS servers with MySQL and Oracle backends.
- Designed centralized, revision controlled management,
monitoring, and capacity planning infrastructure using
git, cfengine, nagios, cacti, and collectd.
- Used puppet to better exchange configuration information and
integrate with other engineering organizations.
- Setup and curated Mediawiki based intranet for engineering.
- Responsible for training and directing lower level sysadmins.
October 2004 - June 2005
Adecco Technical, Inc on contract to Nike, Inc
October 2002 - October 2004
- Eight month staff augmentation contract.
- Managed Solaris 7, 8, and 9 systems to support Nike's B2B
- Supported ATG Dynamo, Weblogic, iPlanet Web Server, and Apache
configurations, and software deploys.
- Built Nagios and RRDtool based monitoring and reporting
system to provide the business with availability and quality
of service reporting.
Senior Systems Architect
Networked Knowledge Systems, Inc
- Responsible (with three other root-level administrators)
for all management functions for over 700 Debian Linux and Solaris
- Centrally managed over 190 Linux iptables/netfilter firewalls
using in-house rule generation scripts.
- Built and supported high-volume apache-based web hosting
environment handling over 16 million hits a day.
- Wrote server virtualization software, using User-Mode
Linux to host up to 30 virtual Linux systems on a single
piece of hardware. Integrates virtual machine creation,
execution, firewall rule generation, and other management
- Wrote a centralized server management system to control
provisioning, backups, monitoring, and firewalling. Reduced
new server provisioning time to under five minutes.
- Designed and implemented geographically fault tolerant mail
processing infrastructure to process several hundred thousand
emails per day, filtering spam and removing viruses for thousands
of users across dozens of companies.
- Weaved together several open source projects to create a
"Spam Quarantine" system which intercepts spam prior to entering
customer systems, and provides them with an easy to use and safe
web interface for recovering and reporting misclassified spam.
- Built a 13 TB online storage automated backup system, reliably
backing up all 700 machines on a nightly basis, with weeks of
incrementals and long term archival of selected backup sets.
May 2001 - October 2002
GMx Technologies, Inc
- Managed small UNIX (Solaris and Debian Linux) and NT (Win2000,
WinXP) network for technology startup.
- Supervised one root level system administrator.
- Implemented a shared UNIX and NT infrastructure
- Configured UNIX, NT, Web, and Jabber (instant-messaging) servers
and clients to use a central LDAP server for authentication.
- Installed and configured NFS, Samba, and DAV servers to provide
users shared home directories and project space.
- Set up small (under 50 customer) virtual hosting environment
- Set up IPSec based VPN system, providing secure remote access and
allowing employees to telecommute.
Supported Services: NTP, DNS, NIS, IPSec, 802.11b (WiFi),
- Performed high-level designs for suite of nine UNIX systems management
- Designed and implemented a Linux-based Appliance operating system for
virtual machines on IBM s390 (zSeries) mainframes under z/VM or LPARs.
- Built "hercules" based build and development environment to simulate
- Built ARM-Linux development environment for Solaris 8 systems.
- Led a team of four developers in the construction of CM_SAFE™,
UNIX configuration management and disaster recovery tool
- Authored and co-authored architectural documents design
specifications, and whitepapers.
- Enforced "dev", "build", "test", and "production" environments,
using user-mode-linux where possible, to allow efficient testing of
- Co-developed iterative project management framework, allowing GMx
to rapidly design and implement new features.
- Built ARM-Linux embedded systems development environment for Solaris
November 1999 - May 2001
Senior Systems Architect/Administrator
2nd Century Communications
Supported Services: NTP, DNS, NIS, CVS, Cfengine, Sendmail, MTI and
EMC disk arrays, Network Appliance NAS, Connect::Direct, rsync, PPTP,
IPSec VPN tunnels
- Grew ad-hoc 8 machine network into a unified 70 machine network
(50 Solaris, 20 Linux) with one other UNIX admin in a "startup"
- Performed team-lead duties, including writing objectives,
prioritizing work, and making purchasing decisions.
- Introduced central management of Sun (Solaris 7, 2.6, 2.5.1) and
Linux (Redhat 6.1, 6.2, Mandrake 7.0, 7.1, Debian 2.1, 2.2) boxen using
cfengine and revision control of all relevant system configuration files
- Built clustered environment where any system could take over the
duties of any other system in under five minutes using SAN and NAS
technology, cfengine, rsync, and in some cases Veritas Cluster Server.
Using Jumpstart, cfengine, and automated restore scripts (Veritas
NetBackup), machines could be completely reloaded and brought back
into service in under 2 hours (and less than 15 minutes of sysadmin
- Managed security.
- Designed firewall, DMZ, and Extranet networks.
- Wrote distribution scripts that integrated CVS with rsync and ssh
to maintain multiple redundant machines on DMZ that heal themselves
in the event of security compromise.
- Deployed light weight intrusion detection software ("snort")
to detect attacks and send alerts.
- Designed and deployed honeypot networks.
- Wrote scripts to lock services in chroot(2)ed "jails".
- Secondary Firewall admin for a pair of Checkpoint Firewall-1
firewalls running on Nokia hardware.
- Implemented system and service monitoring using netsaint, cricket,
and logsurfer. Wrote service checking scripts to do end-to-end testing
of critical services, and perform self-healing or paging if a fault
- Assisted in design and build-out of UNIX equipment supporting
company's core ATM phone switches including call server, network
management systems, and provisioning equipment.
- Supported three terabytes of storage consisting of MTI and EMC
disk (managed by Veritas file system and logical volume manager),
A1000/D1000 disk arrays, and Network Appliance filers.
- Designed and implemented small but highly scalable web farm.
- ..And much, much more! :)
April 1999 - November 1999
Senior Systems Administrator
RO Systems Consultants at Intermedia Communications
Automated network administration of servers and desktops using "cfengine"
and other tools with a focus on creating an easily duplicated, redundant environment
in which any node can take over the responsibilities of any other node.
Duties and accomplishments:
Supported Services: DNS, NIS, Printing, CVS, Cfengine, Sendmail, Sun Cluster,
Fibre Channel disk arrays
- Managed environment consisting of over 120 Sun Solaris and a
handful of Hewlett-Packard HP-UX systems with three other OS admins and
5 application support admins.
- Implemented system monitors (connectivity, and service availablity)
- Maintained CVS repositories to store system configuration
information and track changes over time.
- Implemented software installation and distribution routines to
distribute software to over 120 machines.
- Evaluated fibre channel disk arrays. Implemented two terabyte Data
Warehousing application on an E4500 Sun Cluster using A5000s for shared
- Developed pre-commit tools to verify data before commitment to CVS
February 1998 - March 1999
Senior UNIX Systems Administrator
Contractor at CDI for IBM Global Services deployed at Lucent Technologies
Saint Petersburg, FL
Duties and accomplishments:
Supported Services: Mail, DNS, NIS, NFS, SysV printing, Backups, X/CDE,
Dial-out modem pool
- Owned (with two other root level sysadmins) a small network of 35 HPs
running HP-UX 10.20 and 8 Suns running Solaris 2.6.
- Supported HP OpenView Network Node Manager, Seagate NerveCenter, and
Informix, Sybase, and Oracle databases.
- Wrote fully automated system install scripts for HP-UX and Solaris
- Authored local security policies.
- Planned the migration from HP OmniBack to Vertias Netbackup.
- Built HP and Solaris public domain software environment.
- Built Solaris development environment.
- Planned capacity, suggested high availability solutions.
- Evaluated fileservers, RAID software/hardware, high availability options,
and backup software/hardware.
- Traveled to remote Lucent sites to setup and configure systems.
- Wrote system monitoring software to gauge the health of the systems.
- Managed development and QA environments
- Interviewed and evaluated sysadmin candidates
- Designed and implemented a hierarchal YP/NIS system, allowing user
accounts to be shared among NIS domains.
June 1997 - February 1998
Assistant System Engineer/System Support Engineer
Silicon Graphics / Cray Research
Supported Services: Mail, NIS, NFS, DNS, BSD printing, backups
- Evaluated customer needs to provide hardware and software recommendations.
- Assisted sales representatives in determining customer needs.
- Assisted SGI's customers by debugging system problems and performing
hardware, software, and operating system maintenance.
- Provided customers with software development support.
- Wrote software and scripts for duplicating and managing EFS format
- Designed network layout of new Portland SGI office, including purchasing
recommendations for routers and switches.
- Maintained the Oregon SGI webpage, used to distribute software
and diseminate information to local SGI customers.
- Wrote scripts for and helped debug IRIX autoinst (now RoboInst)
November 1994 - February 1998
UNIX Systems and Network Administrator
Portland State University (School of Engineering)
Portland, OR 97210
Duties and Accomplishments:
Supported Services: Mail, DNS, NIS, NFS, automounter, printing (LPRng), Backups,
X/CDE, Dial-in PPP modem pool, flexlm license managers, X terminals.
- Co-owned (with 3 other root level admins) approximately 110
machines, both Silicon Graphics systems running IRIX 6.3 and 6.2 and
Sun systems running SunOS 4.1.x and Solaris 2.[3-6].
- Supported approximately 4000 users with a small group of root
level sysadmins and several student user-support admins.
- Managed primary DNS maps for 15 sub domains, 27 networks, and
about 5000 devices.
- Mentored incoming junior UNIX admins.
- Centrally managed DHCP and bootp services across 28 subnets in 10
- Operated an NNTP server with a full USENET feed.
- Evaluated fileservers and disk arrays.
- Installed and upgraded software and operating systems.
May 1996 - November 1996
Customer Demo Room Administrator
Intel Corporation - Hawthorne Farms Campus
Hillsboro, Oregon 97124
Duties: Putting together and preparing machines for use as demos for
customers, demonstrating products to customers, testing motherboards,
testing software, hardware and software installs and maintenance,
configuring systems, designing network structure, setting up a Windows
NT server for a 10 machine network, setting up UNIX web servers, and
writing a primitive content management system for facilitating internal
||User-Mode-Linux based virtual machine manager
||Co-founder of TampaBaySAGE, a USENIX affiliated group for system
||Cross-platform central Host Configuration Database |
||Cross-platform software package management and distribution
|Apache and NGINX Webservers, JBoss and Tomcat appservers,
Amazon AWS Services, Automation and APIs, DNS BIND 4.x,
8.x, and 9.x, NIS, RADIUS, backups (Legato Networker,
Veritas NetBackup, AMANDA, OmniBack II, dump, rsync), X
terminals, printers (SysV, BSD, and LPRng spoolers), modem
connectivity (CISCO access servers, Shiva RAS), VPN (Sonicwall,
IPSec, PPP over SSH) sendmail, user accounting, performance
tuning, HP Openview NNM. Linux LVM2, HP-UX, SGI, and Veritas
Logical Volume Managers. Solaris DiskSuite. Cfengine, RCS,
CVS, UUCP, PPP, HTTP, CGI, Sup, Majordomo, Sun Clustering,
Veritas Clustering. Nagios, Splunk, Cacti
|Perl, Ruby, Bourne shell scripting, C. Some python.
|Primarily Solaris (including OpenSolaris) and Linux (Debian and RedHat based distributions). Previous experience with HP-UX, AIX, FreeBSD, and OpenBSD.
|Firewalls (Firewall-1, Linux iptables/netfilter, BSD ipf),
Relational databases (PostgreSQL, MySQL, Informix, Oracle),
NoSQL databases (Redis, SimpleDB)
Portland State University
Portland Community College