Does Your Team Have Great Answers?

16 05 2013

It’s annual review time for my team. With a pivot to making a conversation out of the review session, some really interesting things happened. I’ve been asking a lot of questions and I’ve been getting a lot of really great answers. The results had me on the edge of my seat.

Did your team have good answers to the questions you had for them at review time? If they had great answers, then you’re doing a great job. If they didn’t, then you’re not.

Rands’ excellent post, ‘No Surprises‘ has been my informal guide for reviews this year. I took his, ‘A Review is a Conversation’ to heart, and debated with my team on where they’re headed, their areas of opportunity and the high leverage projects they should be working on.

Funniest thing is, if you’re making your people great, they should have great answers about their careers. And If you’re not asking questions of them at review time, perhaps you’re afraid the answers will fall short.

This year, my entire team had great answers.





Board Room to Back Office

5 11 2010

How quick are you from board room to back office?

Through the four stories I shared in my QCon presentation yesterday, the broad accomplishment of Shopzilla’s greenfield site redesign, was incredible business agility. It’s true that we’re very fast from board room to back office.

I’ll extrapolate a little here on how our continued investment in our engineering and deployment infrastructure has been the biggest enabler for our agility.

Archetypes

Perhaps our biggest enabler has been our Maven archetype. In fact, we no longer have only one archetype, but a myriad archetypes for different purposes; web-services, spring-batch, Java-based command-line jobs.

Our archetypes allow our engineering teams to focus more immediately on solving business problems. Engineers of all levels of seniority, focus less on boilerplate, project structure, and deployment concerns. Our Maven archetype encapsulates;

  • JMX MBean exposure using Spring
  • Metrics exposure, through JMX using JAMon
  • Sample Spring configuration and wiring
  • Patterns forcing separation of configuration from application binaries

Business Agility, Cross Pollination, DevOps

As a result project structures look mostly the same from team to team, making cross pollination easy. Importantly too, our applications are homogeneous from a deployment / DevOps perspective. Distilled, performance of applications deployed to our any environment, can be visualized on our internal dashboards (permanently) in a matter of minutes.

I’ve written about this topic previously on the Shopzilla Tech Blog. The hindsight here is particularly interesting, in seeing how much the archetype approach has allowed agility from a business perspective. The proof: beso.com and more recently, tada.com.





Business Agility, Requirements are King

4 11 2010

Information from my QCon presentation, “Hot Swapping your Engines at 30,000 Feet – War Stories from Shopzilla’s Site Redesign”

War Stories from Shopzilla’s Site Redesign

 

 





Release Rhythm

11 06 2010

Release rhythm is a hot topic for the team right now. Termed another way, the team is aiming at cheap, frequent, reliable releases. It doesn’t seem like it should be hard, but it is.

When I say ‘team’, I’m not referring to engineering and QA alone. I’m referring to the entire team consisting of scrum master, business, engineering and QA.

Shifting Priorities

Our business team has a worthy challenge. They’re armed with an ultra responsive engineering team and a raft of features which are aimed squarely at improving our site SEO and traffic monetization.

We have two week iterations, and oftentimes by the time we’re attacking the final couple of stories, the business priorities have shifted.

Features for the Short Term

With two releases going out per iteration, there’s a tendency to want to ‘squeeze’ stories into the coming release. Stories arriving late into the release branch are often rushed, trend high with defects, and create churn and uncertainty as we’re trying to ship our product. With regularity, it would seem, defects find their way to production.

It’s true that as the team addresses production defects and releases hot-fixes, our velocity for the iteration is impacted. Of course the overarching goal is to be responsive to change while minimizing atrophy through defect churn.

Dev Complete

In our retrospective today, the team engaged in a discussion about ‘dev complete’. It was interesting to observe the discussion, particularly as focus shifted to reviewing test cases, and considering the announcement of ‘dev complete’ to mean production ready.

Mature Code-Base

It’s true our code-base is mature. Over time, it can be difficult to maintain a focus on craftsmanship and constant improvement through refactoring. There were great retrospective items today, where we identified opportunities for refactoring in the context of user stories which had come to pass.

Our Defect Trend


Where To?

  • I’m not sure how it only struck me now: should we consider one week iterations? What with those shifting priorities?
  • Monitoring our defect trend will continue to be critical. We’ve only just started watching this metric in earnest, but surely it can only aid our focus on quality.
  • Can added focus on the pivotal, story by story announcement of ‘dev complete!’ help? The team is sure hoping so.
  • We’re planning to identify brittle parts of our code-base, and prioritize clean up accordingly. We in fact have one story in this vain in the current iteration. Will this help for the long term?
  • Can we shift our expectations from a ‘feature squeeze’ mentality to a releasable feature set (planned in advance) mentality?




Scaling Out the Performance Testing Client Layer

27 04 2010

Rudi Hacopian, Quality Assurance lead on one of Shopzilla’s highest scalability challenges yet, “Inventory LIVE!”, has posted an excellent account of his performance testing challenges (and ultimate successes).

What’s the optimal approach for performance testing an application with;

  • 200 million domain objects
  • 90 JVMs, constituting an Oracle Coherence data-grid, and 12 load-balanced Tomcats
  • RESTful web-services, using FastInfoset, JAXB and streaming over HTTP
  • A core requirement to deliver 100′s of gigabytes of data within a short time frame

In Rudi’s article, “The Web-Service May Not be the Bottleneck!“, he discusses the key challenges inherent with performance testing at such scale, and how scaling at the client layer helped achieve higher network utilization and throughput.

One core goal of the Inventory LIVE! project was to deliver a ‘snapshot’ of our 200 million domain objects to our clients in a short time period. Once the functional testing effort was out of the way, we focused squarely on the complexity of performance testing. I was personally thrilled and excited about our new and unique approach to shipping such an inordinate amount of data to our clients!

With our Coherence grid fully loaded at 200 million ‘mock’ domain objects (we use a mock data loader), we launched multiple clients against the web-service to test the performance and timeliness of our HTTP streaming approach. An hour into the test, I was puzzled and had A LOT of questions for the team. Why had we only streamed 2MM offers so far? Why does it seem so much slower at scale?

Read on!





Fire Chief Role Yields Results

24 04 2010
Twitter post from @uclajug inquiring about the yield of the Fire Chief role

Twitter post from @uclajug inquiring about the yield of the Fire Chief role

It’s been some months since the inception of the Fire Chief role, and I’d like to share further information about how the Fire Chief has helped the Inventory Engineering team realize their maximum potential. The Fire Chief role has proven that where legacy systems cause distractions, removing a single person’s capacity from the team’s total can actual increase the team’s velocity: team members are more focused, and have fewer distractions.

Before the Fire Chief

As I joined the Inventory Team, I was amazed to learn that the daily stand-up was more than 50% focused on discussion about fires. Any net-new user stories included in our iterations saw the attention equivalent to a second class citizen. Fires: important. New, strategic, foundational engineering projects: unimportant.

The Inventory Engineering Team's languishing velocity prior to the Fire Chief

The Inventory Engineering Team's languishing velocity prior to the Fire Chief

After the Fire Chief

I still remember the day when the Fire Chief hat arrived in a wrinkled brown envelope at the office. I must admit to being pretty excited as I sat down with the team to unveil our brand new, bright red, Fire Chief hat. For a brief second there, the perspective that legacy systems and fires make for less glamorous work melted away.

If I had to point to the coolest part of the Fire Chief role? Overhearing people around the office, “Have you seen the Fire Chief?”. It’s so true that our Fire Chief is incredibly popular. What else is cool? Our burndown…

The Inventory Engineering Team's improved velocity with the Fire Chief

The Inventory Engineering Team's improved velocity with the Fire Chief

It’s also worth noting, that in a team where legacy systems meant distracted engineers, and bred more legacy systems, the Fire Chief role has bred focus, a passionate (popular?) team, and excellence. I’m pretty confident when I describe the Inventory Engineering team as “high performing”. Our stand-up meetings (which the Fire Chief still attends) are focused around new projects, with approximately 1 minute of the 15 minute stand-up dedicated to the Fire Chief’s daily report.

Proof in the Pudding

Our strategic, foundational engineering project, “Inventory LIVE!” is currently being launched by the team. Inventory LIVE! is focused squarely the incredibly challenging problem of scale; 200 million domain objects in an Oracle Coherence data-grid, 90 JVMs, 12 Tomcats, and RESTful web-services, using FastInfoset and JAXB, streaming over HTTP.

I should mention that Inventory LIVE! is a net-new engineering project, owned solely by the Inventory Engineering team. I’m confident that such an undertaking would not have succeeded without our Fire Chief concept.





Fire Chief – Cover Man Means Cover Man

12 03 2010

Upon joining a new team at Shopzilla.com, it was fascinating to see how much our myriad legacy systems were impacting our ability to evolve. I am lucky enough to work with a very mature engineer who is passionate about both cross-trained teams, and the evolution of our legacy systems. My recent blog post details how we were able to craft a fun and interesting way to;

  • Improve our legacy systems and services
  • Maintain our production uptime
  • Build a high performing team, focused solely on net-new systems and services
  • Create a fun, light-hearted but professional atmosphere for the above
  • Promote passion for our team and what we do within the organization

My teams deliver high-performing solutions to serious business problems, but they have a ton of fun too! From the article;

On a team with myriad legacy systems, production problems will often be a significant burden for the team. In my experience, without a strategy for managing the team’s approach to tackling these production ‘fires’, the team’s yield for new value creation will be far below it’s potential.

Our approach has been to define a new role – the Engineering Fire Chief. Simply put, our Fire Chief is an engineer (or two) who actively accepts the role of providing distraction-free “cover” for their team. (yes, we actually bought a Fire Chief hat.)

(Keep reading!)

The Fire Chief concept (and hat) is popular.

The Fire Chief concept, and hat are popular.

A fire chief wearing multiple=

A fire chief wearing multiple hats, and proud to fight fires!








Follow

Get every new post delivered to your Inbox.