Macy's Labs

takes the ideas of IdeaLab. Puts them into PRODUCTION.

What are Lean Labs?

*Lean Lab: Image Search pictured

small empowered team who are focused on a particular theme, a particular hypothesis, and they are given autonomy. They are given empowerment to go and test what is best for our customers, to go and run multiple experiments, to test with our customers, come back with the results.

Lean Lab

*Lean Lab #1 pictured

In the CTO's words:
"One of the tools in our labs is what we call lean startups. In April 2014, we kicked off our first lean startup at Macy's, and today we are running twenty-two lean startups. "

Working in "Lean Labs"

Below are some of the problems I was asked to solve while working in the DevOps role of 4 "Lean Labs" at Macys.

"Nick, you own these environments. It's up to you to monitor them and FAST! The duration of this lab is only 3 weeks."

solution:

I wrote this script in python that I named "heartbeats". Performs a variety of health checks as simple as ping, to validating a json api response and even ensuring an sftp login works.

Learn More →

"Nick, the federated cloud environment that the Release Engineering deptartment provisioned for us expired. This keeps happening and we have no control because RE owns these environments".

solution:

Researched and discovered metadata for Time To Live in days before these environments are destroyed (which would cause considerable delay for my team). Wrote a python script to periodically check this value and provide an alert when expiration was near.

Learn More →

"Nick, some of the code our team's developers check into a legacy monolithic code base along with the rest of the enterprise. Sometimes their code is delayed in CI due to a build error and we are not alerted. Can you keep an eye on their specific build jobs and personally ensure their code is merged to master?"

solution:

Researched some endpoints in gerrit and discovered I could use the employees ID number or Name and query each of the git repos to be alerted of their checkins.

Learn More →

"Nick, for our federated cloud environments owned by the RE team, can you deploy the artifacts in the evening after all our team's developers have checked in their code for the day so that the environments are ready in the morning for showcase in our morning standup meetings?"

solution:

The RE team provides a multi-page web portal as a front end for uDeploy and would not share access. I worked with one of their RE contacts and he enabled an API for deployments and here is a pyhon script I used with a json payload to start a deployment in their environments.

Learn More →

"Nick, is there a way you can notify us in Skype when X is complete?"

solution:

python messaging wrapper can be called from any of my script to send messages to the team's chatroom.

Learn More →

"Nick, we need you to trigger a jenkins build at a certain time each night. We don't have administration rights on this jenkins job."

solution:

Without administrator access to the jenkins job we are unable to use the 'build periodically' cron format job scheduling. Scheduled a this python script as a scheduled cron job to kick off a particular jenkins build.

Learn More →

"Nick, sometimes after a certain deployment the version is not updated. Is there a way you can validate the deployment was successful before notifying QA the build is ready for testing"

solution:

python script that uses credentials in the http headers and returns the artifact details.

Learn More →