Few people understand the challenges and pressures facing corporate IT teams — the problems of integrating old and new software, developing new projects and automating processes and (gulp) adopting new methodologies — like the consultants who are often brought in to help them sort things out.
Consultancies often work at the critical fault lines in IT, a location that gives them a unique perspective on all the dynamics that stand between enterprise and quality, cost-effective software. We caught up with Jason Garber, a software engineer who is one of the co-founders of Philadelphia-based PromptWorks, to ask about his experience in helping companies confront these issues.
Jason Garber, who founded his first website development company at age 14, later became an advocate for Ruby on Rails, clean code and automated testing. In the interview, he takes us through the process of rescuing a failed software process, including a reliance on test-driven development and continuous integration tools. He describes the challenges of countless dependencies that seem, to me, to make a good case for the use of methods such as Service Virtualization to help cope with increasingly complex software environments. He might or might not agree, but heck, we’ll let him speak for himself. We’ll present the interview in two parts. Here’s Part I:
Can you start by describing your consultancy and the types of clients you work with?
We create new Software as a Service (SaaS) products, APIs, services, and other software that solves business problems like analyzing data or scaling infrastructure, which enables our customers launch new products or services, scale apps for the next 100,000 users, automate their server infrastructure and eliminate business inefficiencies. We focus on quality and have developed a best-of-breed practices to shorten the development cycle, lower risk and improve outcomes.
Our exceptionally high standards, flexibility, and insight produce quick results and long-lasting returns. We partner closely with our clients, helping them make smart technology choices. Our clients range from startups like IFTTT, PicWell, Clean Markets, ConnectDER and Thrive TRM to enterprises like Comcast, Sungard Availability Services, Sony and American Eagle.
I’m guessing you are often dealing with a crisis when you arrive on the scene. What sort of challenges are you usually up against when you first start working with a company?
Most PromptWorks projects involve some sort of existing software. Either we’re rescuing a failed software project or we’re writing new software that has to integrate with legacy systems and poorly-designed services. Sometimes we run the code through analysis tools just to show that it gets an F. Even though clients have already seen the signs of poor technical or organizational performance, which is why they’ve called on us, it can be a shock when they realize they’ve spent hundreds of thousands of dollars on bad software.
Bad software and APIs mean we have to either rewrite them, gradually replacing them with clean, well-architected code, or write middleware to make the interface consistent and reliable to consumers of the service. As we do that, we use test-driven development (TDD) to accelerate development and ensure proper factorization. We run our tests on a continuous integration platform and continuously deploy internally so stakeholders can always see our progress and give us quick feedback. Getting the client’s developers and IT staff used to such an automated, fast-moving workflow can also be a challenge.
Another challenge we find when we start working for a new client is being at the tail end of a waterfall process, which results in us waiting for dependencies and upstream projects and making elaborate plans and concrete designs. We have better results using Agile and working with other teams in a more vertically-integrated fashion, where each team responsible for delivering some set of user stories has the power to work on whatever needs to be done to complete the story. Enterprises often apply the Agile label to waterfall processes or a very specific systems development life cycle (SDLC) model, but for us the important thing is that we collaborate with the right people to iterate on working software.
Do you see a lot of IT cultural resistance to software business transformations when you go into an organization? Can you describe where that resistance comes from and what causes it?
Clients often call us in because something about the way they build and deploy software is broken, but the remedy is sometimes hard to swallow. Often our client isn’t IT, but IT has to sign off on our choices because they have to support the app in production. Some IT departments aren’t used to cloud-native applications and aren’t accustomed to an Agile process where everyone’s on the same team and pitches in to finish the sprint. When they’ve spent so much energy ossifying around fixed contracts, specialized silos, waterfall, and a cascading delivery schedule, changing the culture is tough.
The resistance we see is mostly an unwillingness to learn something new or take individual risks. IT operations are often on the hook to support applications and services in production, so they have to understand all the technologies developers send their way. With full plates, they don’t want to learn Docker, come to standups, or relax restrictions they made years ago when somebody got burned. If an individual advocates for a technology or process that ends up causing problems, their job is on the line, but if the team fails in the same way they’ve always failed, nobody gets fired.
The danger is that the development teams will bypass IT entirely. Now that it’s so easy for anyone with a credit card to run their own apps in the cloud and purchase managed services for production, IT risks becoming irrelevant. Product management and the revenue side of the business go to bat for the software engineers because they want the product shipped yesterday. IT needs to offer as much and be as good as the marketplace if they want to be seen as a valuable partner rather than an impediment to revenue generation.
What methods have you developed to break down that resistance?
Two things: Compromise and teaching by example.
Clients bring PromptWorks to the table with their IT teams, and we talk about what we can live with and how much extra IT will cost our client by not using the best technologies and processes. Ultimately, it’s up to our client to work it out with their IT department. We just advocate for the smartest choices and support our case with figures and our experience.
The other method we use is to try and bypass their dev-test infrastructure and do our pre-prod work in the cloud using best-of-breed tools. They start to catch on when they see each GitHub pull request has a link to an app running that branch on Heroku and green checkmarks from CircleCI, Code Climate and Hashicorp Atlas. They watch us spin up complete environments in minutes and deploy apps in seconds. Our pipeline is fast until it hits IT, so after a few months of this, IT starts to see their job could be easier and their risk lower if they just copied our stack. They also get pressure from above to turn around faster when we’ve modeled what’s possible.
What is the biggest problem you see with software development, deployment and operations today?
It’s too complex and too fragmented. Though it’s wonderful that we have many more tools at our disposal than we did a decade ago, developers today have to keep track of myriad languages, data stores, open source libraries, APIs, development tools, design patterns, disciplines, and methodologies. The list of SaaS products we buy to support our development activities is huge and there’s another big bunch to support production–monitoring, analytics, error tracing, content acceleration, customer support. Add DevOps to that and you start to wonder how any one person can keep track of it all: containerization, virtualization, orchestration, configuration management, 50 AWS products …
Ruby on Rails rocked the web development world a decade ago by giving us a powerful, opinionated framework that freed up technical cycles to spend more time solving problems in the business domain. You could start writing application code in seconds and anything you needed was already there waiting for you or just a “gem install” away. It became a lightning rod for the best and brightest, especially in startup world, and so the ecosystem around it was phenomenal. It may not be the hot new thing anymore, but arguably every other framework is still playing catch-up to Rails.
Today, we need a rocket-fueled bandwagon that brings together development, deployment, and operations. Heroku’s 12-factor opinionation is powerful and their add-on library is phenomenal, but there’s still a lot of ground it doesn’t cover and a lot of choices and configuration to be made when setting up an app, so many people choose the breadth, unification, and lower pricing of AWS even though the setup is more tedious.
Imagine if you could start an app and every datastore you might ever need was already there, sleeping until your application called it. Error tracking, analytics, monitoring, logging, email, content delivery, security — wouldn’t be add-ons but already baked in, waiting for you to take notice and doing their job even if you didn’t. All would scale automatically, charging you incrementally more as your app became more successful. The technology and price decisions required up front would be few, letting you focus on delivering business value.
The post How one consultancy rescues IT projects that are in the ditch appeared first on Service Virtualization Today.