This post serves as my personal checklist outlining the tasks that Generative AI can potentially assist with within the context of a SaaS platform. It’ll primarily focus on its capabilities in language and text processing:

• Text Summarization (e.g., create executive summaries)
• Text Classification (e.g., is this customer case a feature request or a bug report)
• Sentiment Analysis (e.g., positive or negative views)
• Emotion Detection (e.g., is your customer surprised, joyful, or angry?)
• Text Tagging (e.g., add hashtags)
• Text Translation (e.g., translate from French to English)
• Named-Entity Recognition (e.g., identify real-world people and places in text)
• Text Generation (e.g., for chatbots, automatic replies)
• Retrieval-Augmented Generation (e.g., use your own data with Generative AI)
• Text Similarity (e.g., for semantic search, more-like-this features)
• Question & Answering (e.g., for decision points, content moderation, evaluate ideas)
• Text Transformation (e.g., transform CSV to JSON, transform free-text to a structured format)
• Anomaly Detection (e.g., for quality control)

Remember, these capabilities can be like Legos – you can build impressive solutions by combining them! For instance, first translate all customer comments into a single language, such as English, then group similar comments together based on their topic, and finally summarize the key points of each group for speed reading.

Text Summarization

Turn lengthy documents into concise summaries. Generative AI can extract key points from customer support tickets, emails, contracts, and more, making information easier to read and understand.

• Example 1: Support agents can grasp customer issues faster with AI-generated summaries of requests.

• Example 2: Generate weekly executive summaries by condensing large volumes of customer messages into executive summaries.

• Example 3: Summarize all customer issues to identify common pain points and areas for improvement as input to the product development team.

Information overload is a real problem for many SaaS companies, because it leads to inefficiencies and bottlenecks in workflows. Summarization helps prioritize tasks, gain insights quickly, and make informed decisions.

Text Classification

Classify incoming text data into pre-defined classes. Generative AI can automatically classify incoming support tickets, emails, and other documents into classes or categories defined by you.

• Example 1: Classifying incoming support tickets according to difficulty level (easy, medium, hard) for efficient routing to first or second-line support and thereby enabling faster resolution.

• Example 2: Efficiently route support tickets by automatically classifying them by type (bug report, feature request, or question) and immediately assigning them to the appropriate team.

• Example 3: In a web directory, automatically add new websites to the appropriate categories.

This intelligent and automatic classification of text can accelerate decision-making and efficiency in SaaS workflows.

Sentiment Analysis

Understand the sentiment behind incoming text data. Generative AI can analyze emails, chat messages, social media posts, and other documents to determine whether they are positive, negative, or neutral:

• Example 1: Prioritize customer inquiries based on sentiment. Flag negative feedback for swift resolution.

• Example 2: Monitor sentiment shifts in customer chats during support interactions. Automatically escalate chats that turn negative to a manager.

• Example 3: Analyze sentiment in customer support tickets to identify areas for product improvement or highlight features customers love.

Sentiment analysis empowers businesses to gain insights into their customers’ feelings through automated analysis of text data. This allows for proactive customer service and data-driven decision-making within SaaS workflows, ultimately leading to improved customer experiences.

Emotion Detection

Gain a deeper understanding of customer sentiment with emotion detection. This advanced AI capability goes beyond basic sentiment analysis by pinpointing specific emotions in customer messages, emails, and social media posts. Emotion detection can reveal anger, disgust, fear, joy, sadness, or surprise.

• Example 1: A healthcare chatline can prioritize chats with extremely fearful patients, ensuring they receive prompt and empathetic care.

• Example 2: An e-commerce platform can use emotion detection to identify customers expressing joy about a new purchase, allowing for targeted upsell recommendations of complementary products (e.g., a new game for a joyful gamer).

• Example 3: A FinTech company might proactively reach out to customers expressing anger to understand their concerns and offer solutions, preventing potential churn and fostering stronger relationships.

Emotion detection empowers businesses to understand customer feelings on a deeper level. This valuable insight allows for personalized customer journeys (e.g., tailored product recommendations, empathetic support interactions) and improved interactions within SaaS workflows. By understanding specific emotions, businesses can provide more empathetic support, offer personalized product recommendations, and ultimately build stronger customer relationships.

Text Tagging

Automatically enrich text content with relevant labels. Text tagging goes beyond text classification by allowing Generative AI to automatically assign relevant keywords, labels, or tags to incoming text data. Unlike text classification, these tags are not predefined but are automatically generated by the AI based on the content.

• Example 1: Automatically add one or more tags to incoming customer messages (e.g., billing, technical issue, feature request). This allows support agents to quickly see which topics are covered and route inquiries efficiently.

• Example 2: Based on the tags on a customer message, show the support agent a list of other customer messages with the same tags (e.g., similar billing inquiries). This can help identify recurring issues and find solutions faster.

• Example 3: Social media platforms can automatically suggest relevant hashtags to users when composing posts. This helps users improve post discoverability and reach a wider audience.

Text tagging empowers businesses to organize and categorize large volumes of text content. This can significantly improve information discovery, organization, and analysis within SaaS workflows. For example, customer support agents can leverage tags to quickly find similar inquiries or identify emerging customer issues. When a new tag suddenly appears and grows in usage, it can signal a new trend or problem that needs attention.

Text Translation

Automatically translate text content across languages. Generative AI offers advanced text translation capabilities, enabling real-time, natural-sounding translations between a wide variety of languages. This allows businesses to bridge the communication gap and reach a global audience.

• Example 1: A multinational company can translate employee feedback surveys into a central language for analysis, fostering a more inclusive and informed decision-making process.

• Example 2: A global support center can offer 24/7 world-wide assistance by automatically translating incoming customer requests into the preferred language of the support agent.

• Example 3: An e-commerce platform can leverage text translation to showcase product descriptions and customer reviews in multiple languages, reaching a wider global audience and boosting international sales.

Text translation empowers businesses to achieve several key goals: increased customer satisfaction through multilingual support, improved brand recognition in international markets, and boosted business growth by reaching a global customer base and expanding sales opportunities.

Named-Entity Recognition

Automatically identify and classify named entities in text data. Named entities can include people, organizations, locations, products, events, dates, monetary values, etc. This allows Generative AI to extract key details and improve information organization.

• Example 1: An insurance company might automatically locate and identify any person names (e.g., policyholder, beneficiary) mentioned in incoming insurance claim reports and add them as involved parties in the case.

• Example 2: A customer service chat service might automatically identify what product or service (e.g., specific software, subscription plan) the request is about, so it can be routed to the appropriate team.

• Example 3: An AI startup might use named-entity recognition to remove personally identifiable information (e.g., names, addresses) from their dataset to ensure compliance with data privacy regulations.

Named-entity recognition empowers businesses to extract critical information from text data, automate data entry tasks (e.g., pre-populating forms), and enhance data analysis within SaaS workflows. This can lead to improved efficiency, streamlined workflows, and better decision-making.

Text Generation

This is a core capability of Generative AI: the ability to produce human-like text. Imagine creating chatbots like ChatGPT, but designed specifically for your SaaS company’s needs. These chatbots, powered by Generative AI, could answer customer questions, troubleshoot issues, and even schedule appointments – all fully automated.

• Example 1: Create chatbots that answer customer questions, troubleshoot common issues, and even schedule meetings – all without human intervention. This frees up your support team to focus on more complex customer inquiries.

• Example 2: In a support chat, leverage Generative AI to suggest the next reply in the conversation with the customer. The support agent can then review and edit the suggestion before sending it. This allows for faster and more efficient customer service interactions while still maintaining a human touch.

• Example 3: A social media tool can use Generative AI to automatically create draft social media posts (e.g., Tweets, LinkedIn posts) to promote a new blog post. These draft posts can include catchy headlines and engaging descriptions, saving social media managers time and allowing them to experiment with different creative approaches.

By leveraging text generation, SaaS companies can increase efficiency by automating tasks, improve customer engagement through personalized content, and speed up their core workflows.

Retrieval-Augmented Generation (RAG)

Boost the power of Generative AI with real-world data. Retrieval-Augmented Generation (RAG) allows you to integrate your own data sources with Generative AI models. This empowers these models to access and leverage the most up-to-date and relevant information for specific tasks.

• Example 1: A customer service chatbot can be augmented with your internal knowledge base (e.g., support policies, FAQs) to answer customer questions directly rather than relying solely on its general training data. This ensures responses are accurate and based on your specific policies.

• Example 2: A news platform can leverage RAG to incorporate live news feeds into its AI models, delivering up-to-the-minute summaries of current events.

• Example 3: A sophisticated customer service chatbot can first search an internal knowledge base for an article that closely matches the customer’s query and then use this article when answering the customer’s request.

The ability to integrate your company’s data with Generative AI is a game-changer. By combining these technologies, you can benefit from the freshness of external data while still enjoying the many advantages of Generative AI. This allows for more accurate, up-to-date, and relevant responses tailored to the specific needs of the business.

Text Similarity

Text similarity measures how similar two pieces of text are in semantic meaning. This powerful capability can help SaaS companies organize information and improve user experiences.

• Example 1: Text similarity can power internal search functions, returning relevant documents even if a user’s query doesn’t match the exact keywords, and rank the documents in order of closeness to the user’s query. This can improve the effectiveness of internal knowledge bases and help users find the information they need faster.

• Example 2: Group large collections of documents into thematic categories based on their content similarity. This makes it easier for users to browse and find the information they need.

• Example 3: On an internal support site, text similarity can identify “Related Support Cases” based on semantic similarity. This allows users to find solutions to similar problems without needing to search through countless articles.

By leveraging text similarity, SaaS companies can improve user experience, streamline workflows, and reduce the time spent searching for relevant information.

Question & Answering (Q&A)

Generative AI empowers machines to understand and answer your questions in natural language, just like you would ask a colleague. This allows businesses to automate decision-making processes within workflows and tackle complex tasks that traditional programming struggles with.

• Example 1: Automate decision points within workflows using Q&A. Imagine a customer support workflow where Generative AI can analyze a support ticket and answer a question like “Is this an angry customer complaint?” Based on the answer (yes/no), the ticket can be automatically routed to the appropriate support tier (second level for escalated issues, first level for regular inquiries). This not only increases efficiency but also improves customer satisfaction by ensuring faster resolution times.

• Example 2: Ensure the quality and suitability of user-generated content with Generative AI. You can leverage Q&A to automatically evaluate content and answer questions like “Is this content appropriate for all audiences?”. This allows for faster moderation and identification of potentially harmful content.

• Example 3: A company has crowd-sourced ideas for a new product and asks Generative AI to perform the first evaluation of the incoming ideas and answer questions like “Does this idea align with our product goals?” This initial evaluation helps prioritize high-potential ideas and saves human resources for in-depth analysis.

Q&A empowers businesses to automate decision points in workflows, improve information retrieval, and streamline data processing, leading to increased efficiency and productivity. This allows human experts to focus on higher-level tasks and strategic decision-making.

Text Transformation

Generative AI automates the process of transforming text data from one format to another. This empowers businesses to seamlessly integrate data from various sources, regardless of format, and unlock its full potential within workflows.

• Example 1: A marketing automation platform can leverage text transformation to ingest customer data from various sources (e.g., CRMs, social media platforms), which might be in different formats (CSV, vCard, etc.). The Generative AI system can automatically convert this data into a unified format, enabling streamlined segmentation and targeted marketing campaigns.

• Example 2: A healthcare provider can utilize text transformation to process patient data from disparate sources. For example, converting doctor’s notes into a structured format allows for easier analysis and integration with electronic health records, improving patient care coordination.

• Example 3: A company receives unreliable data from a live source and can then use Generative AI to clean and standardize the format before it’s processed further, ensuring data quality and improving downstream workflows.

Text transformation empowers businesses to unify data from various sources, streamline data processing workflows, and unlock deeper insights from their data, leading to improved decision-making and better business outcomes. This allows businesses to leverage the full potential of their data and gain a competitive edge.

Anomaly Detection

Proactively identify data issues with anomaly detection. Generative AI excels at recognizing patterns within data. This capability allows businesses to leverage anomaly detection to identify unexpected variations or outliers within datasets. This can be instrumental in proactive quality control, flagging potential issues before they impact downstream processes.

• Example 1: A multilingual marketing agency can leverage anomaly detection to ensure the quality of translated marketing materials. The Generative AI system can analyze translated text for inconsistencies or deviations from the original source content, flagging potential errors for human review before campaign launch.

• Example 2: An e-commerce platform can utilize anomaly detection to identify irregularities in product data feeds from suppliers. The Generative AI system can compare new data against historical data, detecting outliers such as uncommon words or missing product descriptions, preventing inaccurate product information from being displayed on the platform.

• Example 3: A company receives a spreadsheet with a million user records that needs to be uploaded to their SaaS platform. Before uploading, Generative AI can check for anomalies within the data, highlighting outliers or inconsistencies that require manual review before upload, ensuring data integrity.

Anomaly detection empowers businesses to proactively identify and address data quality issues, minimize errors, and improve the overall reliability of their data. This leads to better decision-making and reduced operational costs. By proactively addressing data anomalies, businesses can ensure the smooth operation of their workflows and gain a significant competitive advantage.

—ooo—

This glimpse into Generative AI’s capabilities is just a starting point. As this technology continues to evolve, it has the potential to revolutionize various industries and workflows. While it’s important to acknowledge that Generative AI, like any powerful tool, has limitations, it can also generate inaccurate or misleading information at times. As the technology matures, researchers are addressing these challenges. We are only scratching the surface of what Generative AI can do, and the possibilities are truly exciting for the future

The problem with building a business around a thin wrapper of another company’s API is that there is almost zero defensibility against copycats who create a clone of your product. Y Combinator is already funding numerous AI startups pursuing almost identical ideas, and OpenAI has launched their GPT Store, pulling the rug out from under a whole bunch of young AI startups building on top of their API.

If you’ve proven a product-market fit for a new “ChatGPT for X,” you may discover that you’ve essentially conducted free market research for a large incumbent company in that area, such as SalesForce, which can easily add a clone of your product to their portfolio. Alternatively, another startup might simply clone your idea, turning your product into an easily replaceable commodity, driving the price down, and making it challenging to establish a sustainable business.

So, what can a young AI startup do to defend itself against copycats attempting to clone its product?

Proprietaty Datasets

The ultimate strategic asset for an AI startup is to have exclusive access to a valuable and hard-to-copy dataset. In AI startups, the training data is often more valuable than the AI algorithms. For instance, OpenAI, despite its team of world-class AI experts, abandoned robotics because they lacked data.

A less potent version of this strategic asset is to have non-exclusive access to valuable data, such as open-source or public datasets, and then enrich it with additional insights. This can also involve aggregating multiple separate datasets into a unified whole, where the sum is greater than the parts. For example, while Salesforce lacks direct access to HubSpot’s data, an AI startup could aggregate these two data sources to uncover new insights for companies using both systems.

The problem with proprietary datasets is that they are often held by established incumbents, not emerging AI startups. For instance, Google possesses vast amounts of search data, and Tesla has an extensive collection of driving data. In contrast, an AI startup almost always begins with nothing unless it has an exclusive partnership with the owner of such a proprietary dataset.

Niche Specialization

My bet is that most early-stage startup opportunities lie in the application layer of AI, rather than the platform layer. OpenAI is rumored to have invested over $100 million in training GPT-4, indicating that competing at the platform level requires access to massive resources. An exception to this may be found in compliance-intensive domains, such as insurance or conducting business in the EU, where there might be opportunities in the platform layer for specialized or on-premises AI platform products.

While I believe that most opportunities lie in the application layer, pursuing horizontal products—those with broad applicability across various industries—poses a serious risk. You may find yourself merely conducting free market research for an incumbent, as horizontal product ideas often appeal to established companies, such as Microsoft, who may readily clone your idea and integrate it into their existing offerings.

A more effective strategy is to pursue an AI-powered idea specifically tailored to a particular niche—a vertical solution that delivers superior value to a targeted industry compared to any generic, horizontal product. Large players, such as Google or Amazon, often ignore verticals, and even if they do pursue them, they rarely allocate their A-players to these, for them, peripheral opportunities.

Practical ideas for AI-powered startups within a niche or vertical include digitalizing workflows that were impossible to automate in the pre-AI era or developing applications that address pain points that are top priorities for executives in that vertical. Avoid building AI for AI’s sake by solving problems that nobody cares about, even if they can be solved with AI. Instead, focus on solving a problem that truly matters. In B2B startups, the litmus test is always whether someone is willing to pay for your product, rather than merely offering compliments and praise for your innovative idea. Don’t limit your solution to AI, but use whatever it takes, with AI being one tool in your toolbox.

A crucial benefit of the niche approach is that, over time, you can build a proprietary dataset within that vertical, which can be used for fine-tuning generic AI models. This strategy mirrors GitHub’s approach, utilizing its vast code repositories to fine-tune a generic GPT model for its Copilot AI assistant.

It’s important to acknowledge that niche specialization alone does not guarantee defensibility—at least, not until a proprietary dataset has been collected and utilized. This is because a copycat can just as easily attack a vertical product as a horizontal one. While focusing on a niche can reduce the number of competitors, it cannot safeguard against all potential rivals. For instance, you may have built the first “ChatGPT for X,” but then a copycat emerges, learns from your mistakes, and creates a superior version of “ChatGPT for X,” potentially drawing customers away from your original product.

Startup Velocity

A significant advantage that a startup holds over established players is its ability to move quickly. This applies not only to AI startups but to any startup. Until a proprietary dataset has been established to defend against copycats, the key is to rely on the advantage of speed—outpacing incumbents and other AI startups in the same domain. By prioritizing rapid development and delivering a solution that effectively addresses customer needs, startups can potentially enjoy the benefits of first-mover advantages, such as network effects.

Mathematically speaking, a startup cannot achieve instant defensibility. That is, something a small number of people can build in a short timeframe can be copied by competitors with similar or greater resources. If a compelling product-market fit exists, it would be a trivial task for Google to invest the same resources as the startup in pursuing this idea. However, it’s unlikely that they would be able to catch up with a fast-moving startup, especially one with a headstart.

In summary, my rule of thumb for defensibility in AI startups is to focus on a vertical, building a product that people within that niche love to use, and then outpace competitors in terms of innovation and speed to market. Over time, aim to accumulate and leverage a proprietary dataset within that niche, establishing long-term defensibility against copycats and other clones.

Many companies suddenly realized that a major technological paradigm shift had occurred right beneath their feet, and that they needed to move fast and embrace AI technology quickly to remain relevant in the market.

In response to this wake-up call, many B2B SaaS companies have taken the approach of embedding third-party AI chatbots, typically for customer support or other inquiries, as an add-on to their B2B SaaS platforms

But I think this is a too timid response to this transformative opportunity: Customer support and chatbots are not where AI can have its greatest impact on B2B SaaS platforms.

AI is *not* an Add-on Module

Regardless of which vertical or industry a B2B SaaS operates within, its core product consists of supporting business workflows. A workflow is the way that a business generates value, irrespective of the underlying technologies used. Whether a bank employs pen and paper, desktop software, or a SaaS solution, there are certain workflows that they follow to generate value for their customers.

I am convinced that AI needs to be seamlessly integrated into each step of those workflows to optimize their performance. This means not only speeding up the workflows (maintaining the same output levels while reducing processing time) but also enhancing their output (delivering better results with the same cost and speed).

Domain Expertise is the Secret Sauce

The key to the effective use of AI in a B2B SaaS platform is to truly understand the workflows: how they work, how value is generated, and which metrics are important.

Next, map each step of the workflow and examine how AI can enhance or accelerate it. This requires considering the full spectrum of AI capabilities — from text generation and translations to sentiment analysis, text categorization, and summarization.

As an example, consider a scenario where a new insurance claim is reported. AI can analyze the details provided and suggest the type of claim — car theft, lost luggage, or something else — to the claims handler, saving them time and effort in classifying the claim. Additionally, AI can summarize the claims report and suggest an appropriate response to the customer, ensuring prompt and efficient communication. Sentiment analysis can also be employed to evaluate the tone of follow-up messages in the conversation, providing valuable insights for handling customer interactions effectively. This seamless integration of AI can significantly enhance the productivity of claims handlers without compromising human expertise.

AI… Just Another Tool in the Tech Stack

The current perception of AI resembles the early days of the internet in the 1990s. AI is often viewed as a standalone entity, an add-on feature rather than an integral part of the overall application. This approach treats AI as an innovative novelty rather than a fundamental component of the platform’s functionality.

AI should not be considered an isolated feature or module but rather a fundamental tool in our tech stack, designed to excel at specific tasks, much like databases for data storage, email services for user communication, and analytics for tracking product usage. In essence, AI is not a standalone add-on but just another tool that we use to create value for our users.

New Technology, New Problems

Just as the adoption of web technologies brought forth new challenges, such as remote services becoming unavailable due to network outages, which were absent in standalone desktop applications, AI also introduces its own unique set of issues that were previously nonexistent. These include addressing emerging compliance and security concerns, tackling operational challenges like hallucinations, and incurring additional costs.

To conclude, I firmly believe that AI should be considered primarily as a powerful tool to optimize core workflows, the areas where true value is created. Instead of merely adding chatbots as standalone features to handle support inquiries or other non-core tasks, we must fully integrate AI into the core of our platforms to unleash its transformative power.

In the archetypal two-person startup, the non-technical co-founder — typically the CEO — serves as the visionary and dreamer (think: Steve Jobs), while the technical co-founder, the CTO, plays the crucial role of the engineer and builder (think: Steve Wozniak) who brings the startup’s product to life.

However, whether it’s a one-person or two-person startup or a scale-up with a growing executive team, there are core technical responsibilities that must be addressed by the startup. The younger the startup, the more likely these responsibilities need to be handled by the CTO:

• Product: Determining the functionality to build to delight users the most. This can include feature discovery, UI/UX design for new features, and prioritization of product ideas.
• Development: Develop the product that the startup is selling. This includes building the various apps and services comprising the product, such as a native mobile app, backend, and back-office tools. It also involves addressing non-functional aspects, such as security.
• Testing: Ensure the delivered software is of high quality and won’t disappoint users. This involves activities such as automated testing (from unit to E2E), manual/smoke testing (ensuring the feature delights the user), and penetration testing (for security).
• Support: Once the product is launched and users are onboard, support cases will begin to arise, whether due to bugs or missing functionality preventing users from extracting maximum value from the product. This responsibility involves addressing these incoming cases. While often challenging, it’s an incredible opportunity to learn more about users and how they engage with the system.
• DevOps: Manage all the necessary infrastructure (servers and databases) and tools (IDEs, CI/CD pipelines) required to build and run the product.
• Data/Analytics: Provide analytics on how the product is performing on metrics such as Monthly Active Users (MAU) and observe user behavior within it. With the growing importance of AI, the responsibility for data is likely to expand both in scope (including data engineering and data science) and significance.
• Internal IT: Manage tools essential for running the company, but unrelated to software development, such as HubSpot or Google Workspace. Fortunately, this responsibility has significantly diminished over the last 10 years, thanks to the shift to the cloud.

All these responsibilities need not be taken care of by the CTO, especially as the startup enters Series A and beyond. However, they will need to be handled by someone in the startup. For instance, over time, the company might hire a CPO to handle Product responsibility or a CIO to manage Internal IT.

Furthermore, the exact distribution of responsibilities will depend on the composition of the leadership team. For instance, the CEO might take on product responsibilities, or the COO could handle Internal IT.

Many of the responsibilities can be divided. For example, Customer Success can manage first-level support, leaving the CTO to handle second-level support. In testing, the CTO might take care of automated testing (unit tests, service tests, and end-to-end tests), while delegating UI/UX testing and functional testing (ensuring the delivered software meets user needs) to someone else.

My advice is that the most crucial task for a CTO in a startup is to establish a development cycle capable of consistently delivering new features, bug fixes, and refactoring. Building and growing the product should be the top priority for a startup. Following this, consider delegating technical responsibilities not directly related to development, such as Internal IT, to other team members.

A word of warning, though: do not isolate the CTO from customer contact (for instance, by removing them entirely from product, support, and analytics). I firmly believe that the CTO needs customer contact to stay linked to reality. While support may seem mundane and thankless, there is a surprising tendency for startups that take a support-driven approach to development, like Kayak or Wufoo, to perform remarkably well.

So, in summary, if you listen to your customers and develop rapidly and continuously based on their inputs, you can afford to make mistakes in other areas and still succeed as a CTO.

As a CTO at Woba.io, a SaaS startup, I frequently encounter perplexing pieces of advice in my LinkedIn feed. The often low quality of these posts can be attributed to their authors, who are not CTOs themselves but rather software engineers or content writers who are trying to give good advice based on their own personal theories or shallow research, neither of which has been pressure-tested in a demanding real-world environment.

However, I contend that much of the misguided advice and seemingly endless online discussions can be attributed to a simple source: confusion.

The role of a CTO at a startup is dynamic, with success criteria evolving significantly depending on the startup’s stage. Advice that may be entirely logical for a startup poised for an IPO — such as the CTO shouldn’t do hands-on coding — can prove lethal to a seed-stage startup.

In my mental model, I see five distinct stages of startup development, with each stage dramatically altering the nature of the CTO’s role:

1. Seed Stage (Pre-Seed, Seed): In this initial phase, the startup is typically pre-product and pre-revenue. The founders are driven by a promising idea, and the primary objective is to transform this idea into a product that people want, essentially seeking the elusive initial product-market fit (PMF). While the PMF will undergo refinement in later stages, this phase serves as the foundation upon which the journey forward is built.
   • CTO Role: The CTO must be a versatile engineer capable of independently transforming the product idea into a working prototype — an MVP that the startup can present to users and investors to assess their interest. During this stage, expect numerous iterations, based on user feedback, some of which may entail significant changes to the prototype, particularly in the initial iterations.
2. Early Stage (Series A): The startup has successfully launched its product, gained its first satisfied customers, and achieved some initial revenue. The primary focus now shifts towards optimizing the product-market fit to create something highly desirable for a broader audience.
   • CTO Role: At this stage, the CTO will be leading a small, tight-knit team of engineers who are rapidly enhancing the product, moving it ever closer to its optimal product-market fit. Simultaneously, the team must ensure the smooth operation of a live system with real users, underscoring the increasing importance of quality assurance and customer support within the CTO role.
3. Growth Stage (Series B-C): With the startup’s optimal product-market fit firmly in place and a crystal-clear understanding of its potential customer base, significant investments now flow into the startup, all geared toward the rapid acquisition of as many of these customers as possible.
   • CTO Role: The CTO must facilitate explosive user growth, rapidly scaling everything from infrastructure to the engineering team to accommodate this high growth. This involves swift hiring while the CTO’s role also transforms into that of a manager of managers within an ever-growing organization.
4. Expansion Stage (Series D+): The startup, having maxed out organic growth within its customer base, now explores new avenues for expansion. These pathways may encompass global expansion, competitor acquisitions, the launch of new product lines, or the establishment of strategic partnerships.
   • CTO Role: The scope of the CTO’s responsibilities varies depending on the chosen avenue of expansion. For global expansion, it entails product internationalization (beyond mere translation). Acquiring competitors necessitates rigorous technical due diligence and integration into the current platform. The introduction of new product lines involves the incorporation of additional teams into the existing engineering organization.
5. Exit Stage: Eventually, founders and investors will reach a crucial decision — the divestment (selling) of their ownership in the startup, typically achieved through an IPO (e.g., Facebook), a sale to a strategic buyer (e.g., Instagram to Facebook), or a transaction with a private equity firm.
   • CTO Role: At this stage the CTO will typically lead the product/tech workstream part of the exit process, sometimes even in secret, while continuing to do their day job as CTO.

As evident from the above, the role of a startup CTO undergoes profound transformations throughout the startup’s journey, evolving from a one-person powerhouse to a manager of managers, and from a garage-based operation to a fully-fledged enterprise. Alongside these shifting dynamics, the criteria for achieving success in this role also undergo remarkable changes, as do the tips and tricks that can help the CTO becoming successful.

If you’d like to receive an email when the next post, please consider signing up here.

All examples use the same array of blog posts, and each example is self-contained and can be easily copy/pasted (into TypeScript’s Playground.)

Happy coding!

Examples 1: Find the min element in an array using reduce

When we have an array of objects, reduce is an easy way to identify the smallest element. For example, if we want to find the blog post with the least number of likes, we can do it like this:

const posts = [
    {id: 1, title: 'Clean TypeScript', category: 'TypeScript', likes: 91},
    {id: 2, title: 'Stateless React',  category: 'React',      likes: 12},
    {id: 3, title: 'Functional Core',  category: 'TypeScript', likes: 65}
];

const worstPost = posts.reduce((worstPost, post) => worstPost.likes < post.likes ? worstPost : post);

console.log(worstPost.title); // Output: "Stateless React"

Example 2: Find the max element in an array using reduce

When we have an array of objects, reduce is an easy way to identify the largest element. For example, if we want to find the blog post with the most likes, we can do it like this:

const posts = [
    {id: 1, title: 'Clean TypeScript', category: 'TypeScript', likes: 91},
    {id: 2, title: 'Stateless React',  category: 'React',      likes: 12},
    {id: 3, title: 'Functional Core',  category: 'TypeScript', likes: 65}
];

const bestPost = posts.reduce((bestPost, post) => bestPost.likes > post.likes ? bestPost : post);

console.log(bestPost.title); // Output: "Clean TypeScript"

Example 3: Make an array unique using reduce

If we have a non-unique array of objects, we can use reduce to make it unique. For example, if we want to see a unique list of categories used in a number of blog posts, we can do it like this:

const posts = [
    {id: 1, title: 'Clean TypeScript', category: 'TypeScript', likes: 91},
    {id: 2, title: 'Stateless React',  category: 'React',      likes: 12},
    {id: 3, title: 'Functional Core',  category: 'TypeScript', likes: 65}
];

const uniqueCategories = posts
    .map(post => post.category)
    .reduce((categories, c) => categories.includes(c) ? categories : [...categories, c], new Array<string>());

console.log(uniqueCategories); // Output: ["TypeScript", "React"]

The behavior is similar to SELECT DISTINCT category FROM Posts in SQL.

In fairness, using a Set (and not reduce) is a better choice in this use case where we are using an array of values (and not objects):

const posts = [
    {id: 1, title: 'Clean TypeScript', category: 'TypeScript', likes: 91},
    {id: 2, title: 'Stateless React',  category: 'React',      likes: 12},
    {id: 3, title: 'Functional Core',  category: 'TypeScript', likes: 65}
];

const categories = [ ...new Set(posts.map(post => post.category))];

console.log(categories); // Output: ["TypeScript", "React"]

Example 4: Imitate SQL Count with Reduce (Map)

I mostly use reduce as an alternative to SQL’s aggregate functions, such as the one below:

SELECT category, count(*)
  FROM Posts
 GROUP BY category

Which can be imitated using reduce like shown below:

const posts = [
    {id: 1, title: 'Clean TypeScript', category: 'TypeScript', likes: 91},
    {id: 2, title: 'Stateless React',  category: 'React',      likes: 12},
    {id: 3, title: 'Functional Core',  category: 'TypeScript', likes: 65}
];

const categories = posts.reduce((categories, post) => categories.set(post.category, (categories.get(post.category) ?? 0) + 1), new Map<string, number>());

categories.forEach((count, category) => console.log(`${category} (count: ${count})`));
// Output:
// "TypeScript (count: 2)"
// "React (count: 1)"

Example 5: Imitate SQL Count with Reduce (Object Props)

If needing to have the result as object properties instead of a Map, then it can be done like this:

const posts = [
    {id: 1, title: 'Clean TypeScript', category: 'TypeScript', likes: 91},
    {id: 2, title: 'Stateless React',  category: 'React',      likes: 12},
    {id: 3, title: 'Functional Core',  category: 'TypeScript', likes: 65}
];

const categories = posts.reduce(
    (categories, post) => {
        return {
            ...categories,
            [post.category]: (categories[post.category] ?? 0) + 1
        }
    },
    {} as Record<string, number>
);

console.log(categories.TypeScript); // Output: "2"

Example 6: Imitate SQL Count with Reduce (Array of Objects)

If preferring to have the result as an array of objects, then it can done like this:

const posts = [
    {id: 1, title: 'Clean TypeScript', category: 'TypeScript', likes: 91},
    {id: 2, title: 'Stateless React',  category: 'React',      likes: 12},
    {id: 3, title: 'Functional Core',  category: 'TypeScript', likes: 65}
];

type Category = {
    name: string,
    count: number
}

const categories = posts.reduce(
    (categories, post) => {
        const category = categories.filter(c => c.name === post.category);
        if (category.length === 0) {
            return [...categories, {name: post.category, count: 1}];
        } else {
            const c = {name: category[0].name, count: category[0].count + 1};
            return [...categories.filter(c => c.name !== post.category), c]
        }
    },
    new Array<Category>()
);
console.log(categories.length); // Output: 2

// If you want the array sorted descending
const sortedCategories = categories.sort((a, b) => b.count - a.count);
console.log(sortedCategories[0].name); // Output: "TypeScript"

Example 7: Imitate SQL Sum with Reduce

Sometimes it may also be necessary to SQL’s sum function (instead of count):

SELECT category, sum(likes)
  FROM Posts
 GROUP BY category

This can be imitated using reduce like this:

const posts = [
  {id: 1, title: 'Clean TypeScript', category: 'TypeScript', likes: 91},
  {id: 2, title: 'Stateless React',  category: 'React',      likes: 12},
  {id: 3, title: 'Functional Core',  category: 'TypeScript', likes: 65}
];

const categories = posts.reduce((categories, post) => categories.set(post.category, (categories.get(post.category) ?? 0) + post.likes), new Map<string, number>());

categories.forEach((likes, category) => {
    console.log(`${category} (${likes} likes)`)
});
// Output: 
// "TypeScript (156 likes)"
// "React (12 likes)"

Example 8: Add TypeScript Types to Reduce

One of the things I like most about TypeScript is the compiler’s ability to automatically infer types from code; however, if wanting to add explicit types to a reduce function, then it can be done like this:

type Post = {
    id: number,
    title: string,
    category: string,
    likes: number
}

const posts: Post[] = [
    {id: 1, title: 'Clean TypeScript', category: 'TypeScript', likes: 91},
    {id: 2, title: 'Stateless React',  category: 'React',      likes: 12},
    {id: 3, title: 'Functional Core',  category: 'TypeScript', likes: 65}
];

const totalLikes = posts.reduce<number>((likes: number, post: Post) => likes += post.likes, 0);

console.log(totalLikes); // Output: "168"

Hope you liked this post and some of the examples were useful. If you know someone who might like this post, please share it

In other words, instead of calculating the sum of an array of numbers like:

const integers = [1, 2, 3];

let sum = 0;
for (let i = 0; i < integers.length; i++) {
  sum += integers[i];
}

console.log(sum); // Output: "6"

Then, reduce can be used to sum an array of integers like this:

const integers = [1, 2, 3];

const sum = integers.reduce((sum, integer) => sum + integer, 0);

console.log(sum); // Output: "6"

When using reduce, there are three key elements involved:

1. An array with elements that should be reduced to a value, such as the integers array in the example above.
2. An initial value of the variable that will eventually be returned as the result of the reduce function. The initial value is 0 in the sum example above.
3. A function that is called for each element in the array, so in the sum example above it would be called three times. The function has two parameters:
   1. previousValue (renamed to sum in the example above): The accumulated result of the processing so far. When the function is called on the first element in the array, then previousValue has the value specified in initial value (0 in the example above). After that previousValue will hold the return value of the previous element that the function was called for.
   2. currentValue (renamed to integer in the example above): The array element that is currently being processed. The function will be called once for each element in the array.
   3. Return value: The function should return the new accumulated result of the processing. The return value will then be provided in the previousValue parameter when processing the next element in the array.

As can be seen in the documentation there are more optional parameters, but one can get pretty far with the parameters above.

Tip 1: Leave out the initial value

When calling reduce, the initial value can be left out and then the first array element is used as the initial value. For example, to join an array of strings:

const titles = ['Clean TypeScript', 'Stateless React', 'Functional Core'];

const joinedTitles = titles.reduce((joinedTitles, title) => joinedTitles += `, ${title}`);

console.log(joinedTitles); // Output: "Clean TypeScript, Stateless React, Functional Core"

I tend not to use this, because if it is called on an empty array, then it will fail. TypeScript can of course catch those instances on compile time where this might happen but cannot catch those that originate from an external API or database.

Tip 2: Rename the parameters to match your function

The function given as input to reduce is using previousValue and currentValue as default names of its parameters. It’s almost always a good idea to rename these parameters to something more precise and fitting for the domain of the function.

Tip 3: Start with defining the return value

If it is a reduce function with some complex processing, I tend to start with the return value by defining the initial value and then work backwards from that.

Tip 4: Learning by doing

Reduce can be a bit more difficult to learn than map, filter, and the other array functions, and I think the best way to learn reduce is simply to copy/paste some of the examples from this post (and from my post with reduce examples) into TypeScript Playground to get some hands-on experience, learning-by-doing and all that.

In the next post, I will add a number of examples of using Reduce to solve a number of common scenarios. Stay tuned!

Level 1: One Team

My rule of thumb is to have a single engineeering team for as long as possible. In practical terms, this means as long as there are less than 10’ish members of the engineering, I keep it as one team. The reason is to avoid the overhead of having multiple teams and to maximize the benefits of smallness (low overhead, instant communication, fast decisions) for as long as possible.

I don’t really think too much about defining leads or other roles at this level, and I strongly prefer Full-Stack Engineers (or Full-Stack Product Engineers), rather than front-end or back-end developers, simply to be able to move as fast as possible.

Level 2: Multiple Teams

As some point at the end of seed funding or start of Series A, the number of engineers in the SaaS will exceed 10 people, and one needs to start to think of having multiple teams.

My strong preferen here is to use Product Teams, rather than Technology Teams, to align the teams with the product portfolio and with what’s important for the business. For example, I’d prefer to have a car booking and billing team, rather than a frontend and backend team.

It’s kind of like when Uncle Bob talked about Screaming Architecture (in his fine book, Clean Architecture), meaning that when you look at the architecture you should know whether it the architecture for a social media platform, a car rental service, or computer game. I think it’s the same when it comes to team structure: the org chart should scream what this department is developing, not what technologies are used.

Level 3: Team of Teams

At some later point in the scale-up journey, the company may reach a size (in the ballpark of around 100’ish members of engineering) where all teams cannot report directly to the CTO and you need another organizational layer.

For this I much prefer some variation of the so-called Spotify model where related product teams (squads) are group into tribes — and avoid things like SAFe and related methodologies favored by non-technical CTOs. Again, to minimize bureacracy, align the teams with the business success, and maximize speed.

Now, what are your experience with scaling engineering teams?

Even when the CTO is a Head of Development in a SaaS, I still see two different types:

1. One is the Startup CTO whose “development department” consists of a single small team (<10 people) in a young, fast-growing company.
2. The other is the Executive CTO who is responsible for a large development department (>40 people) that consists of multiple teams where typically only managers and/or directors report directly to the Executive CTO.

I often see discussions about what the CTO should do — should the CTO do hands-on coding? —go haywire, because people think of the two different profiles. And what makes good sense for the Startup CTO may not make much sense for the Executive CTO, and vice versa.

In my mind, the greatest differentiater between the Executive CTO and Startup CTO is that the Executive CTO is a manager of managers, so Executive CTO is mostly about scale, and the Startup CTO is mostly about speed.

However, while there are tons of things that differentiate the two types of CTO — and I’ve been both — I think what unites them is that the person will be the last line of defence for all technical issues and decisions in the company, and this is super important!

In a large SaaS you may have a Director of Engineering who is responsible for 30-40 engineers, but can still escalate tough choices to the VP or CTO level, whereas a Startup CTO may have just 5 engineers, but no one to escalate to and must just “deal with it” whatever it is.

This means that whether you’re a Startup CTO or an Executive CTO, then an unusually high degree of resourcefulness will be required to flourish in the job

The story I often hear goes like this: As a reaction to having had a too technical CTO, a SaaS hired a non-technical CTO who can communicate with non-technical stakeholders, enjoys project management, and knows how to deal with people. These are all good things! But their new CTO didn’t really know technical stuff and didn’t come from a developer background — but was a project manager or some sort of business consultant — and due to this lack of technical insights, the CTO has made some terrible technical decisions and/or hires, and the company is now in deep troubles.

The lack of technical insights means that the CTO cannot make informed technical decisions. Even people who are smart, rational, and math-savvy — but not software engineers — often fail when it comes to this. Paul Graham (co-founder of Y Combinator) once gave this advice to non-technical technical leaders: Give up! His reason was simply that a non-technical manager cannot recognize great engineers or great technical work, and that is why they fail.

Having smart engineers who can advice the non-technical CTO about technical decisions also never really works in practice… As Joel Spolsky (co-founder of Stack Overflow) once wrote, it reminded him of someone who doesn’t know how to surf, try to surf, with advisors standing on the beach shouting to the poor surfer what to do, while the surfer falls into the water again and again.

A CTO who is not coming from a developer background is almost always a problem, but in mature, established software houses, it is less of a short-term problem, because the CTO can cruise on for years on the existing architecture and tech stack — and just focus on pumping out new features and not shaking the boat — before it turns into a serious problem (but when it becomes a problem, it does so with a vengeance and needs a a big, nasty, re-platforming project, but that’s a story for another blog post).

In most SaaS scale-ups, even those without complex tech, making the right technical decisions and the right technical hires is critical to the survival of the company. While the company is scaling up, its CTO needs to decide what components from its original startup codebase (which almost always have a fair bit of duct tape and hacks) to keep, what parts to replace, and what to replace them with.

I’ve see at least four developer archetypes who want to influence those decisions, and not necessarily in a way that’s right for the company, and the CTO needs to be able to challenge them:

• Not-Invented-Here (NIH) Engineer: Sometimes as part of the scale-up journey, a company hires a new engineer who concludes that everything that was written before they joined the company is hopeless and beyond repair, so all code needs to be rewritten from scratch. Given that reading code is much more difficult than writing code, this is a common conclusion. For example, I know one company where a senior developer had started a naive project of rewriting the whole codebase from scratch, without really understanding the existing codebase or the size of such a project. Another typical not-invented-here example is an ecommerce scale-up that I talked with who had decided, in 2021, that they needed to write their own web shop software from scratch, because they didn’t think any of the existing solutions fitted with their pretty common needs (!) and now their web shop was missing essential features, like a mobile app, because they were too busy developing and maintaining other features that are available in most standard ecommerce solutions.
• Cloud-Vendor Salesperson: Smooth-talking salesperson from vendors such as AWS or Google Cloud who is trying to push more of their services into the scale-up’s tech stack, and offer all kinds of temptings-sounding offers, like you get Google’s own engineers to work on your product for free, if you just migrate to them. Of course, their aim is to create vendor lock-in so you stay with their services for good.
• New-Tech Engineer: Working with new technologies is super fun, so often there is at least one software engineer in the company who thinks it could be really awesome to use some new, uncommon technology. One example I recall from some years ago is a SaaS where one of the developers was really passionate about Rust (which I totally get!), but Rust didn’t really have any benefits that were business critical to the SaaS, the developer simply thought it would be a cool language to learn and do something in. The problem was that it was almost impossible for the SaaS to hire experienced Rust developers, which made scaling up their company much slower than it needed to be.
• Opposite-Tech Engineer: An engineer who always wants to use the opposite tech of the ones the company is using. If the company uses AWS, they want to use Google Cloud (and if you had used Google Cloud, they wanted to use AWS). If you use Angular, they want to use React; and so on. In most cases, if techs are more or less comparable, then switching tech is just an expensive project that adds little value to the customers while draining the time of the company’s most senior engineers, so there needs to be a pretty good reason to start such a tech migration.

In my experience, then scale-ups need to make so many important technical hires and decisions — where a wrong choice can put the company’s health at risk as neither funding nor time is unlimited — that they need a CTO who can listen to and challenge all of the persons above, and separate signals from noise, and at the end of the day be able to confidently make the right technical decision for their company.

That’s it! Hope you enjoyed this little series on the wrong CTO profiles for a scale-up

1. The CTO was the too technical co-founder
2. The CTO was a non-technical, project manager type

In this post I’ll talk about the first type, and in the next post about the second one.

For the first CTO type, the story usually goes like this: A customer-focused CEO and a technical-focused CTO joined forces and founded a start-up together. And they succeeded!! They discovered a great product/market fit and secured funding to grow to the next level. Regardless of everything else written in this post, then this is pretty amazing! And it’s only something like 1% of start-ups who get this far, so anyone who gets so far has my respect.

But after the product/market fit was discovered, and customers started to rush onboard, and the company got funding to scale up, then the CTO started to struggle with being a technical leader, being responsible for a growing team of engineers, instead of being a brilliant individual engineer.

The situation can often be tricky, because the CTO might lack self-insights of their own strengths and weaknesses, while simultaneously own a significant part of the company, which is only fair, considering the CTO’s instrumental role in getting the company this far, but it means you need to get the CTO onboard before you can make changes to their job.

I’ve seen companies where they tried to move the technical co-founder into a Chief Architect role, without people management, but I must admit that while it sounds like a tempting solution, I haven’t really seen it being that successful in the real world…

Another option, if the company needs it, is to ask the CTO to build a new adjacent product or module from scratch, which is what the CTO has done better than 99% of other founders, and then the company can build a new team around the scale-up’s original product and scale that up. While I’ve seen this work out in the short-term, the problem is often that the company’s focus gets spread over too many products, which is a bridge too far for most scale-ups.

The optimal situation is of course that the CTO gets an Aha! experienceand realizes that being a super-productive engineer isn’t the same as being a technical leader — as the saying goes: what got you here, won’t get you there — and based on this insight does one of the following:

1. The CTO actually wants to become a technical leader (and is fine with designing and coding becoming an ever-smaller part of the job) and wants to get the coaching necessary to build up the necessary leadership skills.
2. The CTO moves to a senior engineer role and reports to a new CTO (often tricky and requires a fairly experienced CTO to make this work out).
3. The CTO simply decides to move on and found a new start-up, if the start-up phase is the part of a company’s journey that they enjoy the most, and then takes another tour in the carousel.

Having had a too technical CTO, some companies overreact by hiring a non-technical CTO, typically a project manager type, who’s good at the non-technical stuff, but in the next post I will explain why this profile is an equally bad choice for a scale-up. Stay tuned!

There is no historical precedent from other professions of working this way. If we look at other professionals who master an advanced body of knowledge, like attorneys or a medical doctors, then they discuss directly with their client, without an intermediate layer of managers or bureaucracy, and listen to their client’s needs and apply their body of knowledge to help solve the client’s problem in the best possible way.

From a supply/demand perspective, then if we look at writing code as an isolated skill, then this is becoming easier and easier to learn (due to easier languages, richer framework, and amazing tooling), which means that the barrier of entry is continually lowered, which increases the supply and competition.

Moreover, the rise of low-code/no-code tools gives even more people the ability to perform coding-like tasks. And a last thing that boosts the supply is that the COVID lockdown taught even the most conservative company that working remotely is feasible, so they can hire in many new locations. Right now we have a hot job market, but historically those things don’t last forever, unfortunately.

So in a world where the supply of generic coding skills increases, how can programmers differentiate themselves from the competition?

I think the key is to learn the business of their company — and then start to think of themselves as, primarily someone who solves business problems, and secondarily, someone who solves these problems by writing code.

How does a programmer create value?

Code has no inherent business value in itself. Code only adds value when it contributes to solving a business problem; for example, automating a task, which previously required expensive manual work, which reduces the operating cost of the company, which increases the bottom line.

Now, it would be easy to say that a Business Analyst / Product Manager should be the one understanding the business and then handover a detailed specification over to the programmer who can then translate the specification into code.

However, in my view, a natural evolution is already happening:

• A company used to have backend developers, frontend developers, and maybe even DBAs.
• This made sense because everything was really complicated; for example, memory leaks in C++ could give even the most chilled programmer gray hairs, and operating an Oracle Database was so non-intuitive that it required a dedicated, full-time specialist to keep it running.

However, everything is becoming easier, from automatic memory management to DBaaS.

Why we got Product Teams and Full-Stack Engineers

During the last ten years, we have seen the rise of Full-Stack Engineers and Product Teams (instead of Technology Teams), because:

1. It’s actually possible to master the full stack, because developing comparable functionality becomes easier and easier as we evolve forward.
2. Because a full-stack engineer can deliver complete features (rather than a “layer” of it), which is insanely important from a business perspective, because this increases speed to market — as there are fewer handovers and less coordination — which means that the feature can get into production faster and generate business value sooner and seize business opportunities before they vanish or are seized by a competitor.

The coming rise of the Full-Stack Product Engineer

I think the next natural evolutionary step after the Full-Stack Engineer role is that the Full-Stack Engineer and Product Manager (or Business Analyst, at first) will slowly start to merge together into a single role, meaning that we end up with a person who can talk with the business or customers to understand their problem, and also write the code to solve the problem. In my view this is a natural state of things that aligns with how other professions with an advanced body of knowledge operates.

I feel the need — the need for speed!

Why is this beneficial? Two reasons: (1) Because we can get an even faster speed to market – from idea to production – due to even fewer handovers and even less coordination – and speed becomes evermore important to businesses in our fast-changing world, and (2) many “micro decisions” need to be taken when writing code and the more an engineer knows about the business context, the better (micro) decisions they can take.

So my recommendation is to really learn the business domain that you are writing code for. Don’t wait for someone to teach you, actively pursue the knowledge. This will both make you more valuable to your business right now, because you can make better “micro decisions” and can communicate more easily. And you will be aligned with the long-term evolutionary trend of software development.

Initially, drag and drop functionality was really tricky to support in HTML and JavaScript due to the document-oriented background of these technologies, but HTML5 introduced a super easy way to add drag and drop functionality to web apps.

Basically, we just need to follow the four steps below to enable drag and drop. The first two steps enable the dragging, and the last two steps enable the dropping: 

1. Set the draggabe attribute on the HTML element that you wish to drag.
2. Add an onDragStart event handler to the draggable HTML element to store its identifier when it’s being dragged.
3. Add an onDragOver event handler to the HTML element that should be able to receive dropped elements.
4. Add an onDrop event handler to the HTML element that can receive dropped element to handle what should happen when a dragged element is dropped onto it.

In the follow sections, we will look into each of these steps. Rather than using a library, such as react-dnd, we will implement the steps in pure TypeScript and React.

1. Set the Draggable Attribute

By default, HTML elements are not draggable, so to make one draggable, you need to set its draggable attribute to true, as shown below:

<div id="d1" draggable="true">Drag me</div>
<div id="d2" draggable="true">Or me!</div>

This ensures that the HTML element can be dragged. Obviously, it’s no fun to drag an element without having anywhere to drop it, but it’s a good start.

2. Add an onDragStart Event Handler

Unless you only have one draggable element, you will need to store an identifier of the element that is currently being dragged, so you can identify the element if it’s being dropped.

For this purpose, we add an onDragStart event handler to the draggable elements:

<div id="d1" draggable="true" onDragStart={handleDragStart}>Drag me</div>
<div id="d2" draggable="true" onDragStart={handleDragStart}>Or me!</div>

The event handler itself can be pretty straight forward:

const handleDragStart = (event: React.DragEvent<HTMLDivElement>) => {
    event.dataTransfer.setData('text', event.currentTarget.id);
}

Here we’re using DragEvent, a synthetic React event, as the type of the argument to the event handler, and specifying that it must be a Div element. Depending on the use case, we can pick a more generic type, such as HTMLElement (event: React.DragEvent), or leave out the template entirely (event: React.DragEvent). Personally, I like the types to be as precise as possible to catch as many errors as possible upfront.

Inside the event handler, we call setData to store the identifier of the dragged element in the dataTransfer object, which can later be read by the HTML element that the dragged element is dropped upon. The first argument (‘text’) given to setData is the type of the identifier given as the second argument, and not a property name.

Tip: Remember that id attributes on HTML elements must be globally unique, not just unique for the element type, so a simple number (such as a numeric primary key from a database) might not be enough in itself to guarantee uniqueness, but you can prefix it with something to ensure uniqueness; for example, task_${task.id}.

3. Add an onDragOver Event Handler

By default, an HTML element cannot receive dropped elements, so we must override this behavior on the HTML element that should receive dropped elements.

This is easily done by adding an event handler that simply prevent the default behavior:

const enableDropping = (event: React.DragEvent<HTMLDivElement>) => { 
    event.preventDefault();
}

Add the event handler to the receiving HTML element:

<div onDragOver={enableDropping}>Drop Area</div>

4. Add an onDrop Event Handler

When an HTML element is being dropped, we want to perform some kind of action. This can be done in an onDrop event handler on the element that receives the dropped element:

<div onDragOver={enableDropping} onDrop={handleDrop}>Drop Area</div>

And add the code to the HTML element that can receive dropped elements:

const handleDrop = (event: React.DragEvent<HTMLDivElement>) => {
    const id = event.dataTransfer.getData('text');
    console.log(`Somebody dropped an element with id: ${id}`);
}

We use the getData method to retrieve the identifier that we set in onDragStart event handler, so we can do something meaningful with the dropped object.

That’s basically all the code we need to enable basic drag and drop functionality in a web app. The code snippets in previous sections are collected into a complete example in the next section.

Example 1: A Minimal Drag and Drop Example

For copy/paste convenience, the complete minimal is listed below:

const handleDragStart = (event: React.DragEvent<HTMLDivElement>) => {
    event.dataTransfer.setData('text', event.currentTarget.id);
}

const enableDropping = (event: React.DragEvent<HTMLDivElement>) => { 
    event.preventDefault();
}
    
const handleDrop = (event: React.DragEvent<HTMLDivElement>) => {
    const id = event.dataTransfer.getData('text');
    console.log(`Somebody dropped an element with id: ${id}`);
}

return (
    <div>
        <div id="d1" draggable="true" onDragStart={handleDragStart}>Drag me</div>
        <div id="d2" draggable="true" onDragStart={handleDragStart}>Or me!</div>
        <div onDragOver={enableDropping} onDrop={handleDrop}>Drop Area</div>
    </div>
);

Example 2: Highlight the Drop Area when an HTML is Dragged Over

A missing feature in the drag and drop functionality in HTML5 is that there is no CSS pseudo selector (like :hover on mouse over) to highlight an HTML element when another element is being dragged over it. For example, like in MacOS, where a folder is highlighted when a file is being dragged over it.

But, fortunately, we can easily simulate this hover-like functionality by using the onDragEnter and onDragLeave event handlers as shown in the example below:

const [ dragOver, setDragOver ] = React.useState(false); 
const handleDragOverStart = () => setDragOver(true);
const handleDragOverEnd = () => setDragOver(false);

const handleDragStart = (event: React.DragEvent<HTMLDivElement>) => {
    event.dataTransfer.setData('text', event.currentTarget.id);
}

const enableDropping = (event: React.DragEvent<HTMLDivElement>) => { 
    event.preventDefault();
}
    
const handleDrop = (event: React.DragEvent<HTMLDivElement>) => {
    const id = event.dataTransfer.getData('text');
    console.log(`Somebody dropped an element with id: ${id}`);
    setDragOver(false);
}

return (
    <div>
        <div id="d1" draggable="true" onDragStart={handleDragStart}>Drag me</div>
        <div id="d2" draggable="true" onDragStart={handleDragStart}>... Or me!</div>
        <div
            onDragOver={enableDropping}
            onDrop={handleDrop}
            onDragEnter={handleDragOverStart}
            onDragLeave={handleDragOverEnd}
            style={ dragOver ? {fontWeight: 'bold', background: 'red'} : {} }
        >
            Drop Zone
        </div>
    </div>
);

That’s it. Good luck with making your web apps more intuitive with super cool drag and drop functionality.

Happy coding!

PS. The code is available at GitHub (https://github.com/kenneth-lange/tsx-drag-and-drop).

This pattern enables us to use functional programming for what it does the best, and to use imperative programming for all the messy, mutable, side-effectful stuff that most applications also need to do. And what makes this pattern super nice is that it provides a clear separation between the two programming styles, so they can peacefully coexist in the same application.

The Functional Core, Imperative Shell pattern is similar to Hexagonal Architecture, Ports and Adapters, Clean Architecture, and similar styles that, a bit simplified, are characterized by two key architectural attributes:

1. There is a core with the core business logic, and a shell that handles interactions with the outside world, such as persisting data in databases or providing UIs to interact with end users.
2. The shell can call the core, but the core cannot call the shell and the core is even unaware of the existence of the shell. This is also known as the Dependency Rule (see, for example, Clean Architecture to learn more about this rule).

An additional architectural attribute in the Functional Core, Imperative Shell pattern is, unsurprisingly, that the core should be written in a functional style, meaning using immutable values and pure functions.

The key objectives in the Functional Core, Imperative Shell pattern is to separate functional code and imperative code, and to minimize the imperative code, so when in doubt whether a piece of functionality belongs in the core or shell, then make it functional and put it in the core.

Book Tip: If you want to use Domain-Driven Design and functional programming (for example, in the Functional Core, Imperative Shell pattern), I can warmly recommend Domain Modeling Made Functional, a truly awesome book

In practice, the core will often be an instance variable in the shell. The shell will call the core with a value and get a new value returned, and then do side-effectful, imperative stuff, like displaying it to a user or storing it in a database, with value returned from the core.

If you’d like to read more about the Functional Core, Imperative Shell pattern, I recommend the links below:

• Functional Core, Imperative Shell
• Boundaries
• Functional core, imperative shell (a Hacker News discussion)
• Functional Core, Imperative Shell in TypeScript (my demo implementation in TypeScript).

Thank you for reading. Hope you will find this pattern useful. If you know someone who might enjoy this article, please share it

For example, imagine a bank that offers both banking and insurance to its customers. The bank has acquired the insurance business, which already had an insurance solution. The customer master data is stored in a CRM system. So if we were tasked with developing a customer self-service portal for the bank, the portal would need to call three different systems (CRM/Bank/Insurance), with three different protocols, and three different security schemes, just to get a basic overview of a customer’s engagement with the bank.

A common way to overcome this imperfect setup is to create a new backend in front of the real backend(s) and then design the perfect API for the frontend.

This is known as the Backends for Frontends (BFF) pattern and an example (using our imaginary bank) is shown below:

The Web BFF in the diagram above can expose a simple GET /customers/{id} REST operation that our frontend can call instead of dealing with the complexity of calling and integrating three different systems.

Beside respecting the separation of concerns principle, by separating our presentation logic from our integration logic, we get many more benefits from using the BFF:

• Call the backends in parallel: We can call the backends in parallel and perhaps respond faster to the end user.
• Filter the responses: We can remove internal or sensitive data, such as an unwanted customer flag, or unnecessary data that just adds complexity to the frontend and drains battery power when parsing on mobile devices.
• Transform the responses: We can transform the responses to the frontend into something more usable, such as translating internal codes to something more descriptive. For example, a code in a job field can be translated from SH_CLERK to Shipping Clerk.
• Enhanced security: We can add extra security in the BFF, such as OAuth2, to protect the unsecure backend solutions, and we can implement a single sign on, so the frontend doesn’t need to deal with different authentication methods in different backend systems.
• Handle different protocols: The BFF can call FTP, SOAP, REST, GraphQL, and other types of services, but still use a single protocol when interacting with the client.
• Encapsulate advanced business logic: Issuing a new insurance policy may be a complex operation that requires multiple service calls, but this can be simplified for the clients into a single end point with only the absolute minimum data in a flat structure.
• Caching across clients: Some of backends may be slow so caching across different clients may be an effective way to deliver acceptable response times to the end users. Moreover, if we need to call usage-based external APIs, it may also be worth caching those responses across clients to reduce the cost.
• Protect the client from changes to the backend: Changes in the backends APIs need not to result in changes in the UI, the changes can be handled in the BFF.
• It’s still just code: A BFF doesn’t involve canonical data models, ESBs, or other old-school integration patterns that are complex and time consuming to deal with; it’s just plain code. Moreover, a change in the BFF will not affect other systems, like with an ESB, so it is better aligned with DevOps and microservice thinking.

The main drawback of the BFF pattern is that, at least in the beginning, it can seem like extra work for the frontend team (which should be the ones developing and owning the BFF) but it usually pays back if the underlying backends are non-trivial. Remember that there are no technical restrictions to what language / framework we can use for the BFF; if the frontend team is skilled in JavaScript, they may be more comfortable with writing the BFF in Node.js and that’s perfectly fine.

For example, a web shop can be designed to scale automatically to handle peak loads, such as Black Friday, but if its core business process depends on an external service that breaks under unexpected loads, then the whole web shop can be brought to its knees, unless we design for this situation.

One of the best ways to address this problem is to use the circuit breaker pattern, which stops all calls to an external service when that service is unavailable and then the circuit breaker returns an artificial response instead (more on that later).

In the Java-based NewsReader example in the previous blog post, we can add a circuit breaker as a decorator around the Feed interface as seen in the diagram below:

A pseudocode implementation of the FeedCircuitBreaker class is shown below:

public final class FeedCircuitBreaker implements Feed {
    private Feed feed;
    private List<Post> cache = Collections.emptyList();

    private boolean circuitOpen = true;
    private LocalDateTime circuitRetry;

    public FeedCircuitBreaker(Feed feed) {
        this.feed = Objects.requireNonNull(feed);
    }

    @Override
    public List<Post> getPosts() {
        // Is it time to reopen the circuit and try again?
        if(!circuitOpen && LocalDateTime.now().isAfter(circuitRetry)) {
            circuitOpen = true;
        }

        if(circuitOpen) {
            try {
                cache = feed.getPosts();
            } catch(Exception e) {
                // Close the circuit and wait 5 minutes before trying again
                circuitOpen = false;
                circuitRetry = LocalDateTime.now().plusMinutes(5);
            }
        }

        return cache;
    }
}

And using the circuit breaker is a matter of decorating an underlying feed:

Feed f = new FeedCircuitBreaker(new TwitterFeed("@KennethLange"));

The tricky part of the circuit breaker pattern is to design the right fallback strategy where we need to decide what will happen when the external service is unavailable. There are many different fallback strategies we can choose from, and some of the most common ones are shown below:

• Crash and burn: We assume errors will not occur and don’t design for it. For example, we have all seen websites where raw PHP errors are shown directly in the HTML returned to the end users (not recommended, neither from a usability nor security point of view).
• Wait and see: Inform the users that the service is temporarily unavailable and encourage them to try again later. Wikipedia has an example of this implemented in PHP.
• Alert an operator: A manual task is created for an operator. For example, to check whether the RSS feed has been moved to a new address.
• Empty response: We return an empty response. For example, if the response from a service is used for displaying product recommendations on a website, then we can return an empty list and let the UI act as if there were no product recommendations.
• Default response: We return a safe default response. For example, in an insurance claims process, we can default the fraud risk to high, to be on the safe side, but otherwise continue with the automated process.
• Cached response: We return the last successful response. Like in the example with the newsreader earlier in this post.
• Degraded response: We use local data to generate a lower quality response as a substitute for the real one. For example, in the insurance claims process in the previous bullet then if it’s a VIP customer who hasn’t reported a claim during the last year and the claim amount is less than $1,000 then we consider the fraud risk as being low.
• Eventual consistency: We will synchronize the systems later. For example, if our web shop cannot access the inventory system, we will still take the customer’s order, put the request in a queue, and update the inventory system when it eventually becomes available again.

The right fallback strategy is almost always a business decision. For example, issuing motor insurance policies based on partial information may be an unacceptable risk and a wait and see strategy should be used. But for a web shop it may be a better fallback strategy to accept the customer’s order, without knowing if we are able to fulfill it, and then send a gift certification if we later realize that we cannot fulfill the order.

That’s it. I hope you enjoy this short post about a design pattern that doesn’t get all the attention that it deserves.

The ports and adapters pattern (also known as hexagonal architecture) is an effective way of addressing these problems. The problem for existing applications is that re-architecting them may not be economically feasible. So a minimalist approach is to use ports and adapters when you call external services, which you do not control, but not necessarily when using external libraries or products, which you do control.

The basic concept behind ports and adapters is very simple: You define a port that can be used with various adapters as long as the adapters conform to the port’s interface. An example from the physical world is the HDMI port in your laptop, which you can connect to any monitor as long as the monitor obeys the HDMI standard.

Moving back to software, then a simplified example of using ports and adapters is shown in the Java-based newsreader below:

The core package contains the core logic of the newsreader. I know that the feedreader will need to connect to various newsfeeds, but I don’t want to pollute my core code with all kind of implementation details, such as storing Twitter client secrets, obeying rate limiting in the API calls, dealing with broken XML in RSS feeds, so I define the Feed interface that defines the methods that I need a feed to offer.

The Feed interface can be something really simple like the code snippet below:

public interface Feed {
    List<Post> getPosts();
}

All the implementation details, I keep in the infrastructure package where I keep the implementations of different news feeds, such as RSS and Twitter. To avoid polluting the core code with implementation details, the key is to use inversion of control, so the core package has no dependencies on the infrastructure package.

When your application needs to connect the core code with the infrastructure code, you can use dependency injection in your application:

// Initialize the feeds from the infrastructure package
List<Feed> feeds = new ArrayList<>();
feeds.add(new RssFeed("http://feeds.feedburner.com/KennethLange"));
feeds.add(new TwitterFeed("@KennethLange"));

// Inject the feeds into the FeedReader
FeedReader reader = new FeedReader(feeds);
for(Post post : reader.getRecentPosts()) {
    System.out.println(post.getTitle());
}

Beside the benefit of keeping the concerns of core and infrastructure code separate, you can add new feeds, such as Atom or Google News, without touching the core code. If Twitter shutdowns their APIs then you do not end up with dead Twitter code scattered throughout your application, and you can provide other feeds that provide similar content.

Another benefit is that your unit tests of the core code become much more reliable as you can easily make a fake (or stub) implementation of the Feed interface for testing purposes. This both means that you no longer need to rely on unreliable infrastructure in your tests and the unit tests will most likely become a lot faster to execute, which will also benefit your continuous integration.

Beyond Java
The basic idea of ports and adapters is that you define an interface and can collaborate with anyone who implements the interface. The concept is not limited to Java or other languages that offer an interface type. You can even define the interface as a service contract, perhaps in Swagger, and can then collaborate with any third-party service that implements the contract. This is similar to how webhooks work and would result in a very loose coupling where you would not even need to redeploy your code to support a new service. The difficulty would obviously be to get third parties to support your service contract.

When you call external services as part of your business processes, it is usually a good idea to stick in circuit breaker to avoid instability from a third-party service to spread to your system, so if the third-party service goes down, then your system doesn’t go down with it. The circuit breaker pattern will be the topic of my next blog post to improve improve the way we use external services in our code.

For example, the Task REST API below, which is taken from my company’s REST API, is a typical example of how the collection pattern looks:

The collection pattern is so widely adopted that even a REST newbie, who has never seen this API before, will be able to guess the description in the Task column based on the content in the Method and Path columns.

The collection pattern is also really smart from a code perspective, because many frameworks have some sort of Active Record implementation on top of the collection pattern, so the framework can automatically wrap the whole REST service in a convenient way.

For example, in the old AngularJS framework they had this wonderful $resource factory that you could simply give the URI of a REST service that followed the collection pattern – and then the $resource factory would automatically figure out the rest (no pun intended!)

var Task = $resource('/tasks/:taskId', {taskId: '@id'}); 

var task = new Task(); 
task.description = "Put a man on the moon."; 
task.assignee = "James Webb"; 
task.$save();

While the collection pattern is really clever and so easy to use – and the best solution for almost all REST services – there are some edge cases where its CRUD approach just doesn’t make sense and other patterns should be considered.

Now it gets controversial…

When the web was the new big thing – and online pet shops were worth hundreds of millions of dollars – you would see web pages with HTML forms like this: 

<form method="POST" action="send_mail.cgi">
    <input type="text" name="subject">
    <input type="text" name="message">
    <input type="submit" value="Send Message">
</form>

While you can long for those innocent days when you could publish code like that on the web and not be flooded with spam; the important thing here is that you can also call the send_mail.cgi script directly and use it as a web service; for example, using the small JQuery script below:

var mail = {
    subject: "Man walks on the moon",
    message: "Armstrong and Aldrin become the first men on the moon..."
};

$.post({
    url: "/send_mail.cgi",
    contentType: "application/x-www-form-urlencoded",
    data: mail, success: function() {
        console.log("Mail sent!");
    }
});

Now, I will argue that send_mail.cgi is a RESTful web service (!) even if it’s a really poorly designed one and a simple POST /mails service would have been a lot nicer!

If you finish the demanding, yet satisfying task of reading Roy Fielding’s PhD thesis, which defines the REST architectural style, you will see that it says nothing about limiting our REST services to CRUD operations and it also says nothing about limiting ourselves to the collection pattern…

In fact, Fielding later wrote a blog post about the use of the POST method in REST, and said, “As long as the method is being used according to its own definition, REST doesn’t have much to say about it.”  And if we read the HTTP specification we can see that it doesn’t limit the use of POST to adding new items to collections – and if we read the URI specification we can see that it doesn’t limit our URI naming to plural nouns…

So POST /send_mail.cgi is OK from a specification point of view and can be considered RESTful…

So what are you saying?

So what am I saying? Is this the sacking of Rome and we can now all go crazy with POST /add-new-order.cgi and GET /find-my-orders.xml. No laws! No limits!

Of course not. I still think that the Collection Pattern is the right choice for almost all RESTful web services – and it should be the default choice for any new RESTful web service – because it’s so widely adopted and easily recognizable by most API users.

However, there are edge cases where it makes sense to use other patterns, such as the Controller Pattern. For example, if I have a REST service for rockets then one does not simply just launch a rocket (or walk into Mordor!) because you need to provide launch codes and the rocket needs to go through multiple stages before actual take-off – and this goes way beyond just changing the value of a field in the resource representation. So for this scenario I would add a controller subresource:

/rockets/43/launch-rocket

When breaking with the Collection Pattern I really like the verb-noun naming of the URI, such as launch-rocket. This is without doubts because I read Code Complete way too many times (!) but also because it makes it obvious to the API user that it isn’t a part of the Collection Pattern. On top of that, remember to add a link to controller subresource in the resource representation to make the API user aware that the subresource exists:

{
    "id": 43,
    "name": "Apollo 11",
    "state": "Ready for launch",
    "_links": {
        "self": {"href": "/rockets/43"},
        "launch-rocket": {"href": "/rockets/43/launch-rocket"} 
    }
}

This meaning of this post isn’t to say that the Collection Pattern is bad. In fact, it’s the right choice for almost all REST APIs and everybody will love you for using it! The purpose is to say that Collection Pattern != REST and you still have some wriggle room for edge cases that doesn’t fit neatly into the Collection Pattern without losing your API’s RESTfulness or the desirable properties that come with this architectural style.

At first this team pattern looks similar to the product team pattern, but unlike the product team pattern there is no engineering manager directly responsible for the team’s work.

This pattern also looks a little like the matrix team as the people on the team report to a manager outside of the team, but unlike the matrix team there is no common theme for the engineers who report to an engineering manager.

That is, a single manager is responsible for a diverse group of engineers working with different technologies (e.g., C++, Android, Node.js) and different products areas (e.g., billing, orders, customers) — and there’s no guarantee that two Android Engineers will report to the same manager or that all people on the Orders product team will report to the same manager.

In this team pattern, the manager is less of a traditional manager who directs and supervises work — and more of a career counsler and coach who can advice the engineer.

In this team pattern the leadership of a team can be split into three separate roles:

• Product Manager: This role is focused on product leadership, such as talking with customers and preparing product road maps.
• Engineering Manager: This role is focused on people management, such as recruitment, career guidance, building culture, finding and fix interpersonal issues.
• Lead Engineer: This role is focused on technical leadership, such as technical best practices, mentoring, and anticipate the future technical needs of the team.

An real-world example of this team pattern is described by Josh Tyler, who is EVP Engineering at Course Hero, in his fine book Building Great Software Engineering Teams; at his company Course Hero, an online learning website, they’ve organized their teams according to their product areas:

On each product team they have a Lead Engineer who takes care of technical mentoring and guidance, define best practices, and make sure they are followed.
The Lead Engineer works closely with the Product Manager to scope projects, prioritize tasks, and give the team the context necessary to be successful.

Where it starts to divert from the product team pattern previous example is that the engineering manager’s role is independent from the Lead Engineers.
The Engineering Manager is responsible for a group of engineers across multiple product teams. The Engineering Manager is no longer involved in technical leadership and
doesn’t supervise the work of the Lead Engineer, but is focused on people management, such as finding and fixing people-related problems.

The roles of Engineering Manager and Lead Engineer are completely independent. So it is possible for a Lead Engineer in one product team to report to an Engineering Manager.
This engineering manager may be part of another product team — as a hands-on engineer — and report to another Lead Engineer for technical matters on that team.

But what do Course Hero do when they have a person who is both talented in people management and technical leadership? They considers those rare people to be precious gems that should be treasured and see them as people with executive potential. But still then, they recommend that the person starts by focusing on one of the two paths (i.e., Lead Engineer or Engineering Manager) and then master the other one later.

Strengths

The self-organized product team shares many of the strengths we have seen in the other team patterns.

But a distinct strength is that splitting the engineering manager role into two opens up for a genuine two-track career model with a track for managerial leadership
and another for technical leadership. This is important as most first-rate engineers are not really interested in traditional management, but are deeply passionate about technology, so in this
team pattern they have a career path where they grow without losing their technical edge. This will also make it easier to attract and retain highly skilled technical people
as you don’t expect them to become a traditional manager (and lose that all-important technical edge) or force them to report to a incompetent manager who overrides
their technical decisions due to authority, and not merits.

It may also make hiring easier, which should not be underestimated in a competitive job market, as finding a single candidate who excels in both technical leadership and people management is really tough.

Weaknesses

At first sight the clear definition and separation of product, technology, and people leadership roles look attractive — as it makes it real easy to figure out who to ask:

• Should you upgrade to the latest version of Angular? Ask the Lead Engineer!
• Do you dream of becoming a Full-Stack Developer? Talk with your Engineering Manager!
• What’s the most important feature to work on right now? Ask the Product Manager!

The risk is that in real life many problems doesn’t fit neatly into one of these categories…

For example, the CEO thinks that the team is not developing fast enough:

• Is that a product thing? Maybe the team is focusing on low-value, low-visible features?
• Is it a engineering thing? Maybe there is too much technical debts or poor design?
• Is it a people thing? Maybe the people on the team lack skills in the technologies used or their motivation is low?
• Is it a mix of these? All of these? Some of these?

This will make the debugging of the problem harder — as the debugging may spans multiple areas of responsibility owned by different people — it will make finding and fixing the problem take longer.

From a manager perspective: The primary weakness is that the manager is often no longer involved in engineering, neither as a lead nor as a individual contributor, and if this is the case then his or her engineering skills can quickly deteriorate as things are evolving pretty fast within computing these days (it’s a Red Queen’s race!) and the manager is in high danger of fast becoming a dinosaur that suggests solutions that made perfect sense… on mainframes! Or thinking that ASP Classic and React are more of less the same… I mean they are just web development frameworks, right?

The knock-on effect of losing one’s technical instinct is that it can lead to poor judgement.

Of course, these risks can be mitigated — especially with good collaboration and some general curiosity about what’s happening in one’s field — but left unattended they may turn into issues.

That’s it! I hope you enjoy this little blog series on team patterns!

It has been slightly delayed, because I got a last-minute invite to present at TCC 2018 (our annual conference for customers and partners) and to host a roundtable there as well. Both great experiences You can see a nice photo on LinkedIn where I’m rehearsing together with Christian (our CEO).

But enough about my everyday work and back to the blog series

This post is about the product team pattern!

At first sight a product team is very similar to a matrix team. You organize a team around a product area and take all the different roles needed to build the product and put them inside this team.

The big difference from a matrix team is that all team members, regardless of their role, report to the same line manager. So it doesn’t matter whether a team member is a designer, frontend developer, backend developer; everybody on the team reports to the same line manager.

The motivation for doing this is to simplify decision making (i.e., the buck stops at the line manager regardless of the functional area) and to encourage people on the team to learn more about the business that the product area serves (as they will work within that area for a long time, rather than on the matrix team, where they are more likely to be moved to a completely new product area).

When Instagram moved away from the technology team patterns, which we saw in an earlier post in this series, they adopted the product team pattern, as you can see in the org chart below:

As you can see in the org chart above, Instagram organized their team around the product areas (such as content creation). To handle general stuff that doesn’t fit neatly into a product area, they added two platform teams: The Core Client team for developing the container (or app shell) that the product teams develop their product areas within, and a Core Infrastructure team that handles servers and other infrastructure.

At Airbnb they have made an interesting variation on this pattern: They let their product teams focus on a specific persona, such as guest or host, instead of more traditional product areas, such as billing or booking. This makes it really easy for Airbnb to establish business-related KPIs for the team.

The line manager in this team pattern is often called an engineering manager to show that it is not a manager for a specific technology area (such as mobile) or a specific discipline (such as QA), but rather a manager responsible for all engineering within a product area.

The product team pattern tends to grow leaders who can bring different disciplines together and make them build a unified product where all the pieces fit nicely together. And it encourages leaders to focusing on building a product that actually solves a business problem. This is also the motivation for many organizations who use this team pattern: It aligns the engineering teams’ success much more closely with the company’s success and it becomes much easier to define a business-related KPI for the team (compared to the technology team pattern).

Another interesting dynamic is that companies, which move away from technology teams and to product teams, is that full-stack developers with a good understand of the product area tend to replace the technical specialists (e.g., experts in one layer of the tech stack) as the rock stars of the development organization.

The reason is that a specialist can only typically build partial features. For example, a Django developer who can develop the backend functionality, but doesn’t not know React so cannot finish the frontend part of the feature.

But the developer will report to a line manager responsible for the whole product area (and not just a single technology) and who is motivated towards shipping whole features, and hence, more likely to reward people who can actually deliver whole features.

This dynamic is further accelerated if the company uses continuous deployment and is in a business where speed to market matters (which is most businesses).

Strengths

An advantage of a product team over a technology team is that the team is much closer aligned with business success. The team will not feel like a success if their product has just flunked a major public review — even if their code is so beautiful that it would make Jon Bentley shed a tear.

In continuation of this, it is also my experience that more and more software engineers no longer really fit the old computer geek stereotype, who just wanted to be left alone and code. Now they also want to see their product succeed in the market and make meaningful contributions towards that. As in the old parable about the three stonecutters, they are no longer satisfied with merely cutting stones, they want to build grand cathedrals!

To an even higher degree than with the matrix team, the product team is more likely to build a unified product and have better collaboration across disciplines and lower the risk of a “us-versus-them” culture.

Another advantage of the product team over the matrix team is that decision making becomes much easier as there is only a single line manager involved when a decision needs to be taken or impediment needs to be escalated.

All this usually really contribute to the team’s ability to iterate very fast and launch new product features quickly; and contributes positive to the team’s autonomy.

A caveat is that many of these strengths may be nullified if the team has strong dependencies outside the team’s control. These dependencies could be organizational (like external reviews or approvals) or technical (like an architecture that is a big ball of mud and any change to the codebase can have side effects anywhere else so all teams must coordinate their work).

Weaknesses

A serious risk with product teams — especially compared to technology teams — is that they may pay less attention to technical excellence.

There can be several reasons for this:

1. Engineers may become too focused on market success at the expense of engineering excellence. Especially if there is a strong and opinionated product manager or customer.
2. Each team may become a silo (reporting to a single line manager) and there can only be so many senior engineers within a single team. So, for example, if the team only has a single junior Android engineer who will make sure that the quality of his or her code is satisfying? Or making sure he or she knows the best Android blogs to follow?

Similarly, there is also a higher risk of code duplication. That is, several product teams may code the same functionality in their individual codebases (which may or may not be a problem depending on your belief system).

Another risk compared to matrix teams is that it may become more difficult to move people between teams. When an engineer changes team in the matrix, he or she continues with the same line manager. But in product team pattern, he or she will change line manager as well, which may be a major change. So there may be more resistance, both the engineer might like his or her current line manager and not want to change and start all over with a new manager. Plus some manager have empire tendencies and may be less willing to “give away” an engineer for good. The consequence of this may be that the company is not allocating its people to its highest priorities or biggest opportunities.

Another risk compared to technology teams is that recruitment may be tougher. It is easier to explain to an Angular engineer that it would be great to be part of an Angular team compared to being part of a Life Insurance product team. Even with the matrix organization you can tell the engineer that he or she will report to a manager well-versed in that area. You can mitigate this by explaining why this area is interesting from a technical point of view or important to society at large.

Some companies brand their product teams (at least in job ads) as full-stack team, and given it is cool to be a full-stack developer, the thinking is that it will be easier to recruit people for a full-stack team compared to a Life Insurance team.

From a manager perspective, my experience (as having been both a development manager and an engineering manager) is that being an engineering manager is a more demanding job. It is not because the job is difficult from a technical point of view, it is just that the responsibility is broader. You will essentially become a mini VP of Engineering for a small development department.

You will also have a more direct impact on the business, which you cannot shy away from. That is, as a development manager (for a technology team) you can say that the product is perfect from a technical point of view and it is not your fault that it cannot sell. Due to the broader scope of the role there are also many more things that can go wrong. You will be responsible for things outside of your primary area of expertise but will be responsible for them anyway.

Finally, as a manager your technical skills will most likely erode faster than in the other team patterns. You will be responsible for multiple technologies, such as backend, frontend, data — and have people management on top of that. The rapid pace of technological progress only accelerates this; for example, you were an expert in AngularJS and then they release Angular 2 and all your hard-earned skills become obsolete. And this is not only happening in one layer of the stack, but on all layers, so keep up with everything can become pretty tough.

Given this broad scope of responsibility for the engineering manager — from people management to technical leadership — some companies keep the product team but split the engineering manager role into two: One person will be responsible for technical leadership (i.e., a lead engineer) and one person will be responsible for people management (i.e., a people manager).

This pattern, where the engineering manager role is split into two, will be the topic for the next post in this blog series. Stay tuned!

A matrix team is a temporary product (or project) team, which is made up of specialists from different functional areas. The idea with the cross-functional nature of the team is to increase collaboration between different functions to create better products and faster releases.

An old-school example of this team pattern is Microsoft Solutions Framework (MSF), which was hot back when Microsoft ruled the (software) world:

A feature team is usually focused on a product area and will typically last for the duration of a product release or longer. For example, in Microsoft you would have multiple feature teams working on Microsoft Excel, and one of them would focus on Excel Macros.

An example of the members of a feature team could a Program Manager (responsible for functional specifications and project management within the team), 4 developers, and 2 testers. The Program Manager will tell the developers and testers which is the most important feature to work on and handle the planning and coordination of the feature team’s work.

However, developers report to a development manager who provides guidelines on how they should do their job (e.g., job descriptions, development process, engineering practices, coding standards) and be responsible for people management (e.g., promotions, training, move to a new team, etc.) — and it is the same for the other roles, so program managers report to a group program manager, and testers report to a test manager.

The underlying idea is that it will encourage cross-functional collaboration when specialists are literally on the same team, but the specialists will continue to report to a functional manager who is an expert in their area of expertise.

Tuning the Matrix

The most common parameters to tune in this team pattern are the influence of the line manager on the team’s work (degree of team autonomy) and the duration of the matrix team (short-lived versus long-lived).

Yammer: Very short-lived matrix teams

A modern variation of this team pattern is used by Yammer, a social network for enterprises. A key difference between Yammer and Microsoft is that Yammer continually deploy new features to production and don’t have major product release like Microsoft used to have for their shrink-wrap products.

Yammer’s developers share the ownership of the entire codebase, so there is no thing such as “my code” or “my module”. Each time a task needs to be performed on the codebase, they establish a temporary team for this specific task. When the task is done the code is released to production and the task force is disbanded and developers are free again to join new ad-hoc team to address new tasks.

The thinking is that this is highly agile, and people will not be limited to work on a single product area but can quickly go wherever there is the biggest need for them.

The development manager is responsible for developers within his or her technical area, such as Ruby on Rails, Java, or React. But the development manager is no longer defining guidelines for how the developers should work, but instead a coach who is focus on turning his or her developers into top-notch experts in their given technology.

Spotify: Long-lived, autonomous matrix teams

At Spotify, an online music player, they take a different approach and encourage long-lived, stable matrix teams (which they call “squads”). Their reasoning is that it takes a long time to master a product area, such as Spotify Radio, and mastery is needed to build an awesome product for their users.

They also empower their matrix teams and give them greater autonomy than many more traditional matrix organizations.

However, Spotify still have line managers (which they call “chapter leads”), but with the important twist that the line manager is also an active member of a matrix team (for example, as a back-end developer) to make sure he or she stay in touch with reality.

Strengths

The primary strength of the matrix team (compared to the technology team pattern from the previous post) is that it fosters much closer collaboration across functional disciplines. Now the developers and testers are part of the same team; especially, if the program manager successfully makes a shared vision for the product area.

On top of that being a cross-functional team means that all the necessary skills will be immediately available within the team, so there won’t be these gaps between the teams which we saw in the technology team pattern. This improves time to market:

It also encourages developers and testers to learn more about the business domain, such as banking or medicine, that the matrix team is working in. This is really useful when it is a highly complex business domain with lots of counter-intuitive business rules. For simple domains, such as social networks or blogging, it may be less important.

It also brings engineers closer to the business and makes it easier to see how they contribute to the success of the company, while they can still continue to seek mastery within their chosen technology and continue to report to a line manager who appreciates and understands their technical work. The line manager also enforces alignment and quality across teams, and the engineers will have a second opinion (and supporter) in case of a powerful and persuasive program manager on the team or they feel uncomfortable with the decisions taken within the team.

The matrix organization also scales well and can be used for delivering very large products. Microsoft released many of their greatest hits, such as Windows and Microsoft Office, using this team pattern. They were even able to compete with young startups, such as Netscape, while using this model. Obviously, Microsoft used some dirty business tricks to win the browser war against Netscape, but they would not have been able to compete with Netscape if they had not been able to keep up with Netscape’s development speed.

Weaknesses

In theory, a matrix team has a high degree of autonomy, but in practice multiple line managers will often enforce controls that limit the team’s autonomy and the team will need to consult with the line managers before trying anything too radical.

There is also a risk that work process inside the team will turn into small waterfalls with extensive handovers between the disciplines inside the team. This can happen when the line manager is not actually part of the team but defines the process that his or her people must follow within the team.

There is also a risk that the line managers may not see the big picture and start to suboptimize for their functional area. For example, the test manager wants to introduce NASA-like quality controls, while the customers are actually happy with the current quality level and is much more interested in getting new features quicker (at the current quality level).

Many developers who worked in a matrix team feel like they have two managers (i.e., the development manager and the program manager) and they often receive conflicting signals about what is important. For example, the program manager says that the developer can skip the unit testing to meet the deadline, but the development manager says that unit tests must be written for all new code — and the developer is caught in the middle. Unclear or overlapping responsibilities are a frequent source of conflict and frustration in a matrix organization.

Decision making related to how the team works may even turn into a lengthy process as multiple line managers may need to be involved in a single decision. For example, the development manager wants to introduce static code analysis (and pay the technical debts it reveals), which should be a pure development activity. But the program manager feels that it will delay the development activities already on the team’s roadmap, so she wants to be involved. The test manager feels that it is an initiative related to quality, and hence, he should have a say in it, and incorporate it as part of an overall test strategy.

To address these weaknesses, some engineering organizations introduced product teams where they would continue to organize teams around product areas to harvest the benefits of cross-functional teams, but to boost the team’s autonomy and speed up decision making, they decided to drop the matrix organization (with multiple line managers) and instead have the all people on a product team — regardless of their specialties — report to the same line manager.

We will look at this pattern in the next post in this blog series on team patterns. Stay tuned!

A technology team is focused on a technical area, such as front-end or back-end, and all members of the team are specialists in that area. So there will be no product managers, QA specialists, or engineers specialized in other technologies on the team. All members of the team report to a manager who is also a specialist in this particular technical area.

For example, it could be a mobile team with a number of mobile developers reporting to a mobile development manager:

When there are multiple technology teams, such as a mobile team and a back-end team, there is usually very little (if any) code shared between the teams. This is because the code is often written in different languages/framework (e.g., Swift versus Python) and exists in different code repositories. The interaction between the teams will often happen through REST APIs or similar.

A real-world example of this team pattern is early Instagram (around 2015) where they had split their engineering organization into three development teams (they eventually moved away from this setup, but more about that in a later post):

The reporting lines in such an organization is based on expertise. In other words, a mobile developer will not report to back-end development manager, but to the mobile development manager.

The managers in such an organization is likely to be a senior engineer who has been promoted into management, and now also takes care of people management on his or her team. The manager may still be writing code (or at least have the ability to do so). It is also likely that the manager will handle project management for the team’s work and coordination with other teams.

The rock star in such an engineering organization is likely be a technical specialist measured by some technical standard, such as mastering all the advanced features of the chosen language, or writing the most elegant, concise code.

Strengths

The primary strength of the technology team is its technical mastery, which is likely to be higher than in any of the other team patterns which we will explore in the following posts in this blog series.

The team’s codebase is likely to be of a high quality and take full advantage of the latest advancements within the chosen technology — and there is likely to be little technical debts.

It can also be easier to recruit technical experts for such a team. For example, if an engineer is intensely passionate about Django then the idea of working in a Django team, reporting to a Django manager and being surrounded by Django engineers is pretty attractive by default.

Finally, the team’s manager is likely to be highly competent in the work that the team performs, which is key to high job satisfaction according to recent research. This also means that the manager can evaluate the engineers based on merit, and not some measure which can be easily faked (like who stays the longest in the office). The manager will also be able to provide detailed coaching, such as how to write better code and be aware when an engineer is ready to be promoted to the next level.

Weaknesses

A common problem for engineering organizations which use the technology team pattern is that time to market (for new features) tends to be slow.

The reason is that one team may finish its part of the feature fast, but the next team might be busy with something else and cannot pick up the task any time soon — as shown in the diagram below:

This means that work queues up between the teams. The cost of a feature may not be higher in actual development time, but it can be very expensive in calendar time.

This problem should not be underestimated. Time to market is hugely important for most businesses. In Lean thinking, unfinished features are expensive inventory that cost the company money, because they don’t generate business value until it’s in production and being used by the end-user.

This team pattern will also nudge you towards phased (or waterfall) development where each team finishes its part of the work before passing it on to the next team. This will discourage iterative development and feedback loops — and when mistakes start to occur due to limited communication between the teams, the handovers are likely to become elaborate and time consuming — and it can even lead to a destructive “us-versus-them” mentality between the teams.

In an attempt to overcome the weaknesses of technology teams, some engineering organizations introduced cross-functional matrix teams. The thinking was that this would lead to improved collaboration between functions (e.g., product management, development, testing) because they would now literally be the same (matrix) team. On top of that, the expectation was also that time to market would improve, because when a matrix team takes on a new feature it will have all the necessary skills inside the team to finish it!

So in the next post in this blog series, we will explore the matrix team pattern. Stay tuned!

You don’t need to network a lot before you realize that software companies are organizing their teams in very different ways.

But after you’ve heard about a few dozens companies, you start to detect patterns. You start to realize that even though there are lots of small variations then their team structures can all be boiled down to a handful of general patterns.

In my experience there are four general team patterns that most companies follow. Yes, they have tweaked them to fit their circumstances, but the overall idea behind the pattern remains the same:

1. Technology Team: The team is formed around a technology, such as Android. For example, a team of mobile developers who build and maintain a mobile app.
2. Matrix Team: The developers report to a Development Manager, but they are “lend out” to cross-functional product or project teams where they do their daily work.
3. Product Team: The team is oriented around a product area, such as billing. It’s cross-functional, but all people on the team, regardless of their specialization, report to the same line manager.
4. Self-Managed Product Team: The team is oriented around a product area. But the management of the team is divided into technical leadership, typically handled by an Engineering Lead on the team, and people management, typically handled by an Engineering Manager outside the team.

My plan is that each post in this blog series will explore one of these team patterns in depth to identify its strengths and weaknesses — and then spice the whole thing up with plenty of examples of top-performing companies who are actually using the team pattern in the real-world.

The blog series will continue over the next month. I’ll add links to the posts in the series below when they have been published:

1. Team Patterns: How to Structure an Engineering Team?
2. Team Pattern #1: The Technology Team
3. Team Pattern #2: The Matrix Team
4. Team Pattern #3: The Product Team
5. Team Pattern #4: The Self-Managed Product Team

You can subscribe to my blog to automatically get an email when the next post is published.

If you have any questions or comments to this blog series, feel free to send me an .

To show this point, let’s take the blog post workflow below and expose it as a RESTful Web Service:

There are lots of ways you can implement this, so in the following sections we will try three different approaches and take a look at the pro’s and con’s of each of them.

Let’s get started!

1. Use an Attribute for the Workflow’s State

The easiest way to model the blog workflow is just to store the workflow state as an attribute on the blog post resource:

{
    "id": 54301,
    "status": "Draft",
    "title": "7 Things You Didn’t Know about Star Wars",
    "content": "George Lucas accidentally revealed that…"
}

If a client wants to move the blog post to a new state, it just updates the status attribute to the desired state.

It’s an easy solution and the popular blogging software WordPress is basically using this approach in their REST API.

But when you start to dig deeper into it, you realized that it comes with some serious drawbacks.

The first is that the front-end engineer who writes the client code is forced to search through the API documentation to see what values can be used in the status attribute. This conflicts with the idea that REST Services should be self-describing and not relying on out-of-band documentation.

But this drawback can be fixed by adding a metadata service where the client can get a list of all legal values for the status attribute.

A more serious drawback is that the engineer also needs to look in the API documentation to see what workflow transitions are possible (i.e. can you jump directly from Draft to Published?) and code all these workflow rules in the client code.

This means that business logic is leaking into the client, so if there are many different types of clients (mobile apps, websites, etc.) then each client will be forced to re-implement the workflow logic in their own code, which is not a cost-effective way to do software development.

But even worse, it breaks the fundamental software engineering principle of Don’t Repeat Yourself (DRY) and it violates the separation of concerns between the client and server, which makes it even harder to maintain and evolve the software.

2. Use Hyperlinks for Workflow Transitions

So what should you do if you have transition rules in your workflow, but you don’t want all the bad stuff I mentioned in the previous section?

An alternative approach is to model each workflow transition as a subresource and let clients to use HTTP’s POST method on these subresources to perform the transition. This is inspired by the action pattern in PayPal’s API Standards.

On top of the subresources, you add hyperlinks in the response to let the client know what workflow transitions are possible in the current state.

So with this approach, the response will look like this:

{
    "id": 54301,
    "status": "Draft",
    "title": "7 Things You Didn’t Know about Star Wars",
    "content": "George Lucas accidentally revealed that…",
    "_links": {
        "sendToReview": {
            "description": "Send to Review",
            "href": "/posts/54301/review"
        }
    }
}

The smart thing is that the _links section is automatically updated to show what workflow transitions are available in the current state. So in the example above, you can see that the blog post is in the Draft state, and from there you can make the Send to Review transition to move the post to the Review state.

So if you call POST /posts/54301/review, you move the blog post to the Review state, and then the server will update the _links section to show what workflow transitions are possible in this new state:

{
    "id": 54301,
    "status": "Review",
    "title": "7 Things You Didn’t Know about Star Wars",
    "content": "George Lucas accidentally revealed that…",
    "_links": {
        "publishPost": {
            "description": "Publish Post",
            "href": "/posts/54301/publish"
        },
        "rejectPost": {
            "description": "Reject Post",
            "href":"/posts/54301/reject"
        }
    }
}

The benefit of this solution is that clients no longer need to implement workflow logic in their own code, which means that business logic is no longer leaked into them.

It also reduces the risk of poorly constructed links in the client code — which is a frequent cause of defects in REST clients — because the clients get the links from the server.

Another really cool thing is that if the client needs to display a Next Action menu, it can simply loop through the values in the _links section and use them as menu items.

Finally, it also fits nicely with REST’s goal of self-discovery and HATEOAS.

The drawback compared to the approach in the previous section is that the response is bigger, because it includes the _links section. Another drawback is that the interaction between the client and server has become a little more chatty, because you now need two requests if you want to update a blog post and send it to review.

But I think both of these drawbacks are pretty minor compared to what you get out of it.

A more serious concern is that if you have an advanced workflow, you might end up with a massive number of subresources (i.e. one for each workflow state), which might look a bit messed up.

Another concern is that the server will need to know all states at design time to create subresources for them. This won’t be a problem for most workflows, but if you offer sophisticated workflow functionality where users can customize the states and transitions to fit their special needs it could be problematic.

3. Use a Subresource for Workflow Transitions

So how do you model customizable workflows?

For inspiration, let’s take a look at the issue-tracking tool JIRA, which allows (admin) users to configure their own workflows at runtime. How do they expose this in their REST API?

On their issue resource, they added a transitions subresource where the client can get a list of possible transitions from the issue’s current state. The client can then take one of these possible states and make a POST to the same subresource to transit to that state.

I like their approach, but think it’s a little naughty that they use the same subresource for two different things (i.e. list potential states, and change the current state).

So to use this approach for our blog post workflow, you can add a subresource with potential transition changes (you could also call it “transactions“ or “actions“ depending on your preferences):

GET /posts/{id}/availableTransitions

For a blog post in the Review state, it will return something like this:

[
    {"transition":"Publish Post"},
    {"transition":"Reject Post"}
]

If you want to do a transition, you grab one of the possible transitions from the array and POST it to the transitions subresource:

POST /posts/{id}/transitions

{
    "transition": "Publish Post"
}

A cool thing about this approach is that if you need more advanced workflow transitions, you can add more attributes to the transition subresource. For example, when you transit to the Review state, you might also want to specify a reviewer and a comment:

POST /posts/{id}/transitions

{
    "transition": "Send to Review",
    "reviewer": "Han Solo",
    "comment": "Plz review this faster than you did the Kessel Run!"
}

Another interesting possibility is that if the history of transitions is important (for audit), you could enable a GET method on the transitions subresource to get a full list of all transitions that have been performed on the resource.

You could also decide to execute the transitions asynchronous by returning a 202 Accepted status code and a link where the client can poll the latest status. This could be useful in a money transfer between banks where the actual transfer happens in a nightly batch.

Prakash Subramaniam even goes as far as playing with the idea that you should drop PUT all together, and only allow changing resources through a transition subresource. The good thing is that it neatly separates the interface into a query and command part (as per the CQRS pattern) and you have a strong audit trail of what has happened to the resource.

The drawback is that there are (many) scenarios where it’s total overkill to perform a transaction. For example, to edit the title of a blog post. A live blog post editor would probably end up doing so many transactions that it would overwhelm any kind of history log. But for something like a bank account, it makes good sense to make each update inside a transaction, so you have a complete audit trail.

So which one should I pick?

So we came to the unavoidable question: What approach is the best one?

As always, it depends on the context, but here are some quick guidelines:

So use the Einstein rule (i.e. make things as simple as possible, but not simpler) and start with the first approach, and only consider number two or even number three if there is an undeniable need for them.

That’s all for now. Thank you for reading!

A result of this is that Developer Experience (DX) — which is all about using User Experience (UX) techniques to make life easier for third-party developers calling your public APIs — is fast becoming essential to remain competitive in the digital economy.

But how in the world do you apply UX techniques, such as personas, prototypes and usability testing, to the developer experience?

I personally think this is a really exciting topic and I’ve read hundreds of good articles about the topic, but if I had to choose the best of the best, I would pick these four:

1. Why API Developer Experience Matters More Than Ever — This is the best intro that I have seen to developer experience.
2. User Personas for HTTP APIs — A collection of example personas for a REST API. It is really an eye opener to see how many different types of users there can be of a single API, and how different their needs are.
3. Patterns of Developer Experience — Great post about the principles and patterns for effective DX. The DX Pattern collection at the end of the post is a must-have reference library.
4. Building Effective API Programs: Developer Experience (DX) — If you want to go all the way and create a Developer Program then this article lists what such a program should include and gives an example of what it looks like in the real world.

Happy reading!

Image credit: Tomek Paczkowski

The framework greatly simplifies front-end development work, increases developer productivity, and it’s even fun to program in!

What’s so different about Angular 2.0?

The next major version of Angular will not just be an upgrade, but a complete rewrite of the entire framework, and it will not be backwards compatible with previous versions of Angular.

It’s never a popular decision to break backwards compatibility, and the Angular community has not hold themselves back from telling Google this… repeatedly!

So why did the “don’t be evil” folks at Google decide to do it anyway?

The short story is that the web has evolved a lot since Angular was conceived in 2009, so there are a lot of new web standards (e.g. ECMAScript 6, TypeScript, Web Workers, Web Components) that Angular v1 is not using…

So instead of trying to force these new standards into an old framework, the Angular team thought it would be better to redesign the framework from scratch to truly embrace the new standards and reap all the benefits (e.g. speed of development, better support for large code bases, faster applications) and at the same time also rethink some of the less attractive elements of the framework (e.g. Angular v1 has 5 different ways to model a service and nobody really understands why).

Goodbye MVC! Hello Components!

One of the really big changes in Angular 2.0 is that it’s no longer based on the MVC architecture, but has moved on to a component-based architecture.

What does that mean?

In a component-based architecture you vertically divide your application into (UI) components. For example, in Facebook the Timeline could be one component, and the Chat Sidebar could be another one.

The idea is that each component contains all the stuff you would normally put in the different parts of the MVC.

Components can also be nested. So the Timeline component could be a root component, and then under it there could be a component for showing the posts on the timeline, and another component for showing a box to post new messages to the timeline.

Beside components, there are also modules for grouping a number of related components. For example, to represent a functional area within your application.

Finally, there are services (not to be confused with web services) that provides advanced functionality to the components. For example, handling communication back and forth with a REST Service.

Our Simple App

To get some hands-on experience, I thought it’d be fun to re-implement the Movie App in Angular 2.0. I have previously implemented the same app in Oracle JET, and Sandeep Panda originally coded it in Angular 1.3.

The Movie App is a simple CRUD web app where you can maintain your movie database. While it’s obviously a demo app, which could easily be beaten by an Excel spreadsheet, I think that for demo purposes it has several interesting features:

• A Single Page App (SPA) with multiple states (i.e. pages).
• Reuse of functionality across states.
• Passing parameters between states.
• Integration with RESTful web services.

You can see a screenshot below of the Movie App implemented in Angular 2:

I have used Twitter Bootstrap for the (minimal) UI. The simple reason is that Material Design (which is Google’s CSS/UI framework) for Angular 2.0 is still in alpha state, which is still a bit too early for me… Yes, I’m a chicken

Getting Started

I used Angular 2.0 Release Candiate 5, which was the latest version at the point of writing.

For writing the code, I used the (free) Visual Studio Code editor, because it has excellent support for TypeScript, which is the default language for Angular 2.0.

If you want to run the Movie App example from this post, just follow these steps:

1. Download and install Node.js and NPM (if you don’t already have them). I’m using Node.js 4.5.0 and NPM 2.15.9
2. Download Angular2-movie-app.zip and extract it to a local folder.
3. Open a command prompt and go to the angular2-movie-app folder.
4. Run npm install in the command prompt.
5. Run npm start in the command prompt.

And the Movie App should be automatically opened in your browser.

Designing the Movie App in Angular 2.0

The code structure of the Movie App is shown below:

/app
  main.ts
  movie.component.ts
  movie.module.ts
  movie.routing.ts
  /movies-overview
    movies-overview.component.ts
    movies-overview.component.html
  /movie-creator
    movie-creator.component.ts
    movie-creator.component.html
  /movie-editor
    movie-editor.component.ts
  /movie-viewer
    movie-viewer.component.ts
    movie-viewer.component.html
  /shared
    movie.ts
    movie-data.service.ts 

The app is placed in a folder called app and inside it there’s a main.ts class as the entry point to the application.

There is also the movie root component (movie.component.ts), a routing file (movie.routing.ts) that handles navigation between the components, and a module (movie.module.ts) for storing the components.

There are four subfolders for components (i.e. /movies-overview, /movie-creator, /movie-editor, and /movie-viewer).

The /shared subfolder is for stuff that’s used by several components. In this case, it’s the movie.ts file, which is a class that represents a movie, and movie-data.service.ts, which is a service class that handles communication with the REST service.

The Movie Class

The first step in creating the Movie App is to create a Movie class (movie.ts):

export class Movie {
  _id: number;
  title: string;
  releaseYear: string;
  director: string;
  genre: string;
}

This step was not needed in Angular v1, but the benefit of explicitly defining the class (and its properties) is that it gives the IDE the information it needs to provide auto-completion, compile-time checking and other cool stuff.

Calling a REST Service

The next step is to create the MovieDataService class (movie-data.service.ts), which handles communication with the Movie REST Service.

import { Injectable } from '@angular/core';
import { Headers, Response, Http } from '@angular/http';

import 'rxjs/add/operator/toPromise';

import { Movie } from './movie';

@Injectable()
export class MovieDataService {
  private moviesUrl = 'http://movieapp-sitepointdemos.rhcloud.com/api/movies';

  constructor(private http: Http) { }

  getMovies(): Promise<movie[]> {
    return this.http.get(this.moviesUrl).toPromise().then(response => response.json() as Movie[]).catch(this.handleError);
  }

  getMovie(id: number) {
    return this.getMovies().then(movies => movies.find(movie => movie._id === id));
  }

  private post(movie: Movie): Promise {
    let headers = new Headers({'Content-Type': 'application/json'});
    return this.http.post(this.moviesUrl, JSON.stringify(movie), {headers: headers}).toPromise().then(res => res.json().data).catch(this.handleError);
  }

  private put(movie: Movie) {
    let headers = new Headers();
    headers.append('Content-Type', 'application/json');
    let url = `${this.moviesUrl}/${movie._id}`;
    return this.http.put(url, JSON.stringify(movie), {headers: headers}).toPromise().then(() => movie).catch(this.handleError);
  }

  delete(movie: Movie): Promise {
    let url = `${this.moviesUrl}/${movie._id}`;
    return this.http.delete(url).toPromise().catch(this.handleError);
  }

  save(movie: Movie): Promise {
    if(movie._id) {
      return this.put(movie); 
    } else { 
      return this.post(movie); 
    }
  }

  private handleError(error: any) {
    console.log('An error occured: ', error);
    return Promise.reject("error message: " + error);
  }
}

The code itself is pretty straight forward. It provides some CRUD methods to call the REST service, and each method returns a promise, so service calls can be asynchronous.

But I really missed the nice $resource service in earlier versions of Angular where we got the same functionality in just a few lines of code:

angular.module('movieApp.services', []).factory('Movie', function($resource) {
  return $resource('http://movieapp-sitepointdemos.rhcloud.com/api/movies/:id', { id: '@_id' }, {
    update: { method: 'PUT' }
  });
});

It’s not a big deal to write the service class by hand, it just makes the road towards the code you actually want to write a little bit longer. But hopefully a $resource replacement is on the way for Angular 2.0.

Develop a Component

Now that we have the Movie and MovieDataService classes ready, we can start developing components.

Let’s take a closer look at MovieCreatorComponent (Movie-Creator.component.ts), which is used for adding new movies to the app.

But before we dive into the code, let’s have a quick look to see how the component looks from a UI perspective:

From a technical point of view, the component is just a class with the @Component decorator (kind of like annotations in Java):

import { Component } from '@angular/core';
import { Router } from '@angular/router';

import { Movie } from '../shared/movie';
import { MovieDataService } from '../shared/movie-data.service';

@Component({
  templateUrl: 'app/movie-creator/movie-creator.component.html'
})
export class MovieCreatorComponent { 
  movie: Movie = new Movie();

  constructor(private router: Router, private movieDataService: MovieDataService) { }

  saveMovie() {
    this.movieDataService.save(this.movie);
    this.router.navigate(['/movies']);
  }
}

In the constructor, we say that a router (for navigating to other components) and a movieDataService (for calling the REST Service) should be injected into the component, and stored in two private variables, which we don’t need to declare explicitly.

In the saveMovie methods, we save the movie using the REST Service, and navigates back to the movie overview.

In the @Component decorator, we use the templateUrl property to say what template should be used for the component.

You can see the content of movie-creator.component.html below:

<div class="form-group">
  <label for="title">Title</label>
  <input type="text" [(ngModel)]="movie.title" class="form-control" id="title" placeholder="Movie Title Here"/>
</div>
<div class="form-group">
  <label for="year">Release Year</label>
  <input type="text" [(ngModel)]="movie.releaseYear" class="form-control" id="year" placeholder="When was the movie released?"/>
</div>
<div class="form-group">
  <label for="director">Director</label>
  <input type="text" [(ngModel)]="movie.director" class="form-control" id="director" placeholder="Who directed the movie?"/>
</div>
<div class="form-group">
  <label for="genre">Movie Genre</label>
  <input type="text" [(ngModel)]="movie.genre" class="form-control" id="genre" placeholder="Movie genre here"/>
</div>
<div class="form-group">
  <input (click)="saveMovie()" type="submit" class="btn btn-primary" value="Save Movie"/>
</div>

The template is basically HTML with a couple of Angular extensions:

We use [(ngModel)] to bind an HTML input field to a property in the component class, so that the property in the component class is automatically updated when a user enters something in the input field.

For example, [(ngModel)]=”movie.title” will bind HTML input field to the title of the movie property in the component class, so when a user enters a title it is automatically stored in movie.title property.

In the same way, we use (click) to bind an HTML button to a method in the component class.

For example, (click)=”saveMovie()” makes sure that when a user clicks the “Save Movie” button then the component’s saveMovie() method will be automatically called.

In my opinion, the component approach feels really nice to work with, but it’s hard to explain why, it just feels “brain-friendly”. Maybe it’s because the code structure follows the UI structure you see on the screen.

Conclusion

My first impression of Angular 2.0 was that I was surprised by the upfront costs (e.g. installing npm packages, setting up configuration files) compared to the earlier versions where I just linked to a CDN and then I was ready to start coding.

On top of that, I also needed to create classes and hand-code the calls to the REST Service, which also felt like a hassel when your fingers are aching to start coding all the fun stuff!

But once you have made this initial investment, it starts to pay off (easier to rename classes, typos in the code are spotted instantly, really nice and fast auto-completion), which was extremely helpful when I started coding the components.

The components were the most positive surprise. It just felt like a much nicer way to structure the code compared to the old MVC approach.

If you want to get started with Angular 2, I can recommend the architecture overview document for a quick overview of the framework, the 5 Min Quickstart to learn how to install the framework from scratch, and the Tour of Heroes tutorial as a great way to learn a lot about the functionality that the framework offers.

His list of achievements is staggering: He restored order in post-Revolutionary France. He abolished feudalism and serfdom, established legal equality and religious tolerance, and created an Egalitarian system based on the highly controversial idea that government jobs should go to the most qualified!

He was a military genius who reinvented the art of warfare, won some forty battles, and came dangerously close to total domination of Europe.

So how did Monsieur Bonaparte achieve all this?

The most important thing to remember about Napoleon’s rise to power is that it would never have happened, if it hadn’t been for the French Revolution.

Napoleon was a nobody from Corsica, a Mediterranean island that had recently acquired by France, and in pre-Revolutionary France only men from the French aristocracy were allowed to rise to the upper echelons of the military.

But fortunately for Napoleon (and less fortunate for the French aristocrats!) the revolution broke out and it presented Napoleon with a once in a lifetime opportunity to break through the glass ceiling.

He seized the moment and supported the revolution from the beginning. His brilliant talent for winning battles meant that he quickly rose through the military ranks as he beat the Royalists again and again.

His success as a military man gave him political clout. He used his influence to first become consul, then consul for life, and finally nothing less than “His Imperial Majesty the Emperor of the French”.

So what can we learn from this?

The first lesson is that skills are everything, and without skills you are going nowhere.

Napoleon used all his pre-revolutionary career to build his incredible military skills. Even if the French Revolution had never happened, Napoleon could still have had a decent (but less memorable) career by growing his talents into incredible strengths.

The second lesson is that turbulent times always bring new opportunities with them, because they shake up the status quo.

Yes, turbulence is often frightening (as the French Revolution without doubts was) and many people will be paralyzed by fear and be involuntary pulled along by an unstoppable force of change. But Napoleon seized the changes, jumped to the front and let the forces push him forward.

So what can we learn from this? That you should just take it easy, enjoy the good life, and hope that opportunity will someday come knocking on your door?

No, my friend!

It’s true that the future is so hard to predict that it’s almost impossible to know what opportunities will come by or when. To paraphrase Monty Python: “Nobody expects the French Revolution!”

But it’s also true that turbulent times almost always bring new opportunities with them.

So in turbulent times you should rely less on long-term planning, but instead learn to lookout for opportunities. For opportunities are easily missed because, as Edison said, they are often dressed in overalls and look like hard work.

So what should you do while scanning for opportunities?

You should get to know your talents and practice them until you reach mastery, like Napoleon did with his military talents. So when the opportunity arrives, you know exactly how a person with your strengths should chase it and you can seize it with everything you got!

Now, as you might know, one of Napoleon’s more recent honors has been to be immortalized in an ABBA song. But wait a minute, you might think, wasn’t this song about his defeat and surrender at Waterloo?

How did an extraordinarily gifted man, who followed the first two lessons to the letter, end up being utterly defeated by his enemies?

The short version is that Napoleon’s brilliant victories in the first part of his career gave him a false sense of invincibility and this made him arrogant and sloppy. He started wars that he should have stayed out of, and he no longer applied his tactical genius with the same vigor as in his earlier battles.

So the final lesson from Napoleon’s amazing life is that when you have become successful, you must be wary of the arrogance that success often breed, and remember that “all glory is fleeting” as they used to say to victorious generals in Ancient Rome.

The real power of the Swagger standard comes from the ecosystem of powerful tools that surrounds it.

For example, there’s Swagger Editor for writing the Swagger spec, Swagger Codegen for automatically generating code based on your Swagger spec, and Swagger UI for turning your Swagger spec into beautiful documentation that your API users will love to read.

Why use Swagger?

But why not use another standard (like RAML) or simply open your favorite word processor and start hitting the keys?

There are 5 good reasons for using Swagger:

1. Industry Standard: Swagger is the most widely adopted documentation and specification standard for REST Services. This means that it’s already used in real production APIs, so you don’t have to be the beta tester. It also means that the API user has probably already experience with Swagger, which dramatically reduces the learning curve.
2. Designed for REST: Swagger is really easy to use, because it’s a single-purpose tool for documenting REST Services. So most of the complicated things, like security or reusing resource definitions across several methods, are already handled gracefully by the standard.
3. Huge Community: There’s a great community around Swagger, so when you face a problem, you can usually just Google the solution.
4. Beautiful Documentation: The customer-facing documentation looks really nice. Plus there is a built-in way to actually call the services, so the API user won’t need to use an external tool to play around with the services, but can just do it inside the documentation.
5. Auto-generate Code: You can auto-generate client and server code (interface part) based on the Swagger spec, which makes sure that they are consistent. You could even make your own tools.

How to get started with Swagger?

To start writing a Swagger spec, you simply open the online Swagger Editor and start writing according to the Swagger specification.

You can see a screenshot of the Swagger Editor below. You write your spec in the left-hand side, and you can see the resulting documentation in the right-hand side:

For this post, I’ve created a Swagger specification for the Movie REST Service, which Sandeep Panda developed as part of his post on Angular’s $resource.

If you want to play with the example I use in this section:

1. Open the Swagger Editor.
2. Open the “File” menu, and select “Import URL…”
3. Enter http://www.kennethlange.com/resources/movie_swagger.yaml in the box.

Now let’s walkthrough the example spec!

Part 1: General Information

The first thing that you will notice is that Swagger is written in YAML, which is a format that is very easy to read — even for non-technical people.

In the top part of the Swagger specification, you write all the general stuff about your API:

swagger: '2.0'

############################################################################
#                            API Information                               #
############################################################################
info:
  version: "v1"
  title: REST API for 'The Movie App'
  description: |
    The is a demo Swagger Spec for the sample REST API used by The Movie App that Sandeep Panda developed as part of his great blog post [Creating a CRUD App in Minutes with Angular's $resource](http://www.sitepoint.com/creating-crud-app-minutes-angulars-resource/).

host: movieapp-sitepointdemos.rhcloud.com
basePath: /api

Here is an explanation of some of the properties:

• swagger: This is to say we use Swagger 2.0. It should always be “2.0”.
• title: The title of your API documentation.
• description: A description of your API. It is always nice with examples.
• version: The version of your API (remember that for APIs a low version number is always more attractive, because a high number indicates an unstable interface and hence an extra burden on the clients using it.)
• host: The server where your REST API is located.
• basePath: The path on the server where your REST API is located.

Part 2: REST Services

In the middle part, you define the paths and HTTP Methods.

I have only included PUT below, but you can see the rest in my Swagger file.

############################################################################
#                                     Paths                                #
############################################################################
paths:
  /movies/{id}:
    put:
      summary: Update a movie
      consumes:
        - application/json
      produces:
        - application/json
      parameters:
        - in: path
          name: id
          type: number
          description: The id of the movie you want to update.
          required: true
        - in: body
          name: movie
          description: The movie you want update with.
          required: true
          schema:
            $ref: '#/definitions/Movie'
      responses:
        200:
          description: The movie has been successfully updated.
          schema: $ref: '#/definitions/Message'

Below paths you define a path (e.g. /movies/{id}) and then you define the HTTP methods (e.g. PUT) that the path can be used with.

• summary: A short description of the service. There is also a description property for a more lengthy description, if necessary.
• consumes: The content type of the data that the service consumes (you can have multiple types). The most common is application/json.
• produces: The content type of the data that the service produces (you can have multiple types). The most common is application/json.
• parameters: The different parameters that the service accepts. It is both parameters in the HTTP header, URI path, query string and HTTP request body.
  • in: Where is the parameter located? In the path, in the body, in a header, or somewhere else?
  • name: The name of the parameter.
  • type: The data type of the parameter. The common types are number and string.
  • description: A short, user-friendly description of the parameter.
  • required: Is the parameter required or optional?
• responses: The possible responses that the service can return.
  • (HTTP Status Code): You first specify the HTTP Status Code (e.g. 200).
    • description: A short description of when this response happens.
    • schema: A definition of the response object (see next section for details).

Part 3: Resource Definitions

In the last part of the Swagger spec, you have shared resource definitions.

Given that the movie resource representation is used in almost all methods, it makes sense to write the resource definition in a single place and reuse it across the methods.

############################################################################
#                               Definitions                                #
############################################################################
definitions:
  Movie:
    type: object
    properties:
      _id:
        type: number
        description: A unique identifier of the movie. Automatically assigned by the API when the movie is created.
      title:
        type: string
        description: The official title of the movie. 
      releaseYear:
        type: string
        description: The year that the movie was released.
      director:
        type: string
        description: The director of the movie.
      genre:
        type: string
        description: The genre of the movie.
      __v:
        type: number
        description: An internal version stamp. Not to be updated directly.

Below definitions you define a resource type (i.e. Movie) and then you define its properties below:

• type: The data type of the property. The common ones are string and number. The advanced types are objects and arrays.
• description: A description of the property.
• properties: If the data type is an object, you specify the object’s properties below.

If you need to define complex JSON objects, you can be inspired by the great examples found in Swagger Editor. You can find them by opening the “File” menu, and select “Open Example…”

How to turn your Swagger spec into API Documentation

Once your Swagger spec is stable — and your REST API is operational — you can publish your Swagger spec as customer-facing documentation.

For this purpose you can use Swagger UI, which converts your Swagger spec into a beautiful, interactive API documentation (you can see an online example here).

You can download Swagger UI from here. It is just a bundle of HTML, CSS and JS files, which doesn’t require a framework or anything, so they can be installed in a directory on any HTTP server.

Once you have downloaded it, you put your swagger.yaml file into the dist directory — and open index.html and change it to point at your swagger file instead of http://petstore.swagger.io/v2/swagger.json.

Then you can open index.html in your browser, and see your new beautiful, interactive API documentation:

That’s it! Now you have learned all the basic elements of Swagger. Don’t forget to read Swagger specification if you really want to become a Swagger expert.

Data Storage: Tables versus Resources

Both SQL and RESTful Web Services are centered around data.

In SQL, data is normally stored in tables, but in REST Services it is stored in resources.

For example, in a database you could have a customer table:

SQL> SELECT * FROM customer;

ID FIRST_NAME LAST_NAME OCCUPATION
-- ---------- --------- -----------
1  Luke       Skywalker Jedi Master
2  Leia       Organa    Princess

In REST Services, you would have a /customers resource instead of a customer table.

For example, if you want to get all customers (similar to the SQL statement above), you do it like this:

GET /customers

The response to this request would be a JSON array with an object for each customer:

[
  {
    "id": 1,
    "firstName": "Luke",
    "lastName": "Skywalker",
    "occupation": "Jedi Master"
  },
  {
    "id": 2,
    "firstName": "Leia",
    "lastName": "Organa",
    "occupation": "Princess"
  }
]

CRUD Operations

The HTTP methods, which are used for RESTful Web Services, map neatly to the common SQL statements:

The following sections will explain each of them in more details.

Create

To create a new customer, you use the INSERT statement in SQL. For example:

INSERT INTO customer (first_name, last_name, occupation) 
     VALUES ("Han", "Solo", "Smuggler");

In REST, you create a new customer by sending a POST request with the new customer as a JSON object:

POST /customers

{
  "firstName": "Han",
  "lastName": "Solo",
  "occupation": "Smuggler"
}

Read

To read data in SQL, you use the SELECT statement.

For example, to get a complete list of all customers, you simply call:

SELECT * FROM customer;

The corresponding HTTP command is GET, which you can call like this to the same result:

GET /customers

If you want to lookup a specific customer using the primary key, you would do it like this in SQL:

SELECT * FROM customer

WHERE id = 2;

In REST you would append the id to the REST resource:

GET /customers

/2

But what if you want to lookup something using a non-primary key?

In SQL you would just add a WHERE clause to your SELECT statement:

SELECT * FROM customer

WHERE first_name = “Luke”;

In REST, you append a query parameter to the GET statement:

GET /customers

?firstName=Luke

Note: The specific query parameters available depend on the REST service you are using.

You may want to limit the number of fields returned by a query, because you don’t need to display all the fields, or because you want to improve performance.

In SQL, you just specify what columns should be returned:

SELECT

first_name, last_name

 FROM customer;

In REST, you request a partial response:

GET /customers

?fields=firstName,lastName

Note: Partial responses are not available in all RESTful Web Services, but usually in those where performance is key. For example, mobile apps that may need to operate in an environment with limited bandwidth.

Update

If you want to update all columns on a customer via SQL, you use the UPDATE statement:

UPDATE customer
   SET id = 2, 
       first_name = "Leia", 
       last_name = "Organa", 
       occupation = "General"
 WHERE id = 2;

In REST, you do the same by using the PUT method:

PUT /customers/2

{
  "id": 2, 
  "firstName": "Leia", 
  "lastName": "Organa", 
  "occupation": "General"
}

But what if you only want to update some of the fields?

In SQL you simply limit the fields to those you want to update:

UPDATE customer
   SET occupation = "General"
 WHERE id = 2;

In REST, you use the PATCH method:

PATCH /customers/2

{
  "occupation": "General"
}

Note: The difference between PUT and PATCH is that PUT must update all fields to make it idempotent. This fancy word basically means that you must always get the same result no matter how many times it is executed. This is important in network traffic, because if you’re in doubt whether your request has been lost during transmission, you can just send it again without worrying about messing up the resource’s data.

Delete

If you need to delete a customer, you use the DELETE statement in SQL:

DELETE FROM customer WHERE id = 2;

Similar, in REST you use the DELETE method:

DELETE /customers/2

That’s it! This is my attempt to map the key concepts in RESTful Web Services to the corresponding key concepts in SQL. If you understand these, you already got a pretty good headstart towards learning REST Services.

What had started out as a convenient way for research labs to connect with each other suddenly exploded in size. Jakob Nielsen estimated that between 1991 and 1997 the number of web sites grew with a staggering 850% per year each year!

This incredible growth worried some of the early web pioneers, because they knew that the underlying software was never designed with such massive amount of users in mind.

So they set out to define the web standards more clearly, and enhance them so that the web would continue to flourish in this new reality where it was suddenly the world’s most popular network.

One of these web pioneers was Roy Fielding, who set out to look at what made the internet software so successful in the first place and where it was lacking, and in his fascinating PhD dissertation he formalized his findings into six constraints, which he collectively called REpresentional State Transfer (REST).

Fielding’s observation was that if your architecture satisfies these six constraints then it will exhibit a number of desirable properties (like scalability, decoupling, simplicity), which are absolutely essential in an Internet-sized system.

His idea was that the constraints should be used as a checklist to evaluate new potential web standards, so that poor design could be spotted early, and way before it was suddenly deployed to millions of web servers.

He successfully used the constraints to evaluate new web standards, such as HTTP 1.1 (where he was one of the principal authors) and URI (where he was also one of the authors). These standards have both stood the test of time, despite the immense pressure of being essential protocols on the web and used by billions of people each day.

So a natural question to ask is that if following these REST constraints lead to such great systems, why only used them for browsers and web sites? Why not also create web services that conform to them, so we can enjoy the desirable properties that they lead to?

This thinking led to the idea of RESTful Web Services, which are basically web services that satisfy the REST constraints, and are therefore well-suited for Internet-scale systems.

So what are these 6 REST constraints?

1. Client-Server

The first constraint is that the system must be made up of clients and servers.

Servers have resources that clients want to use. For example, a server has a list of stock prices (i.e. a resource) and the client would like to display these prices in some nice graphs.

There is a clear separation of concerns between the two. The server takes care of the back-end stuff (data storage, business rules, etc.) and the client handles the front-end stuff (user interfaces).

The separation means that there can be many different types of clients (web portals, mobile apps, BPM engines, etc.) that access the same server, and each of these can evolve independently of the other clients and the server (assuming that the interface between the clients and server is stable).

The separation also seriously reduces the complexity of the server, as it doesn’t need to deal with UI stuff, which improves scalability.

This is probably the least controversial constraint of REST as client-server is so ubiquitous today that we almost forget that there are other styles to consider (like event-based protocols).

It important to note that while HTTP is almost always used when people develop RESTful Web Services, there is no constraint that forces us to use it. We could use FTP as the underlying protocol, if we really wanted. Even though intellectual curiosity is probably the only good reason for trying that.

2. Stateless

To further simplify interactions between clients and servers, the second constraint is that the communication between them must be stateless.

This means that all information about the client’s session is kept on the client, and the server knows nothing of it (so no cookies, session variables, or other naughty stuff!) The consequence is that each request must contain all information necessary to perform the request (i.e. it cannot rely on any context information).

The stateless constraint simplifies the server as it no longer needs to keep track of client sessions, resources between requests, and it does wonders for scalability because the server can quickly free resources after requests have been finished.

It also makes the system easier to reason about as you can easily see all the input data for a request and what output data it resulted in. You no longer need to lookup session variables and other stuff that makes the system harder to understand.

In addition, it will also be easier for the client to recover from failures, as the session context on the server has not suddenly gotten corrupted or out of sync with the client. Roy Fielding even goes as far as writing in an old newsgroup post that reliance on server-side sessions is one of the primary reasons behind failed web applications and on top of that it also ruins scalability.

So far nothing too controversial in the constraints. Many RPC implementations could probably satisfy both the Client-Server and Stateless constraints.

3. Cache

The last constraint on the client-server communication is that responses from servers must be marked as cacheable or non-cacheable.

An effective cache can reduce the number of client-server interactions, which contributes positively to the performance of the system. At least, from a user’s point of view.

Protocols, like SOAP, that only uses HTTP as a convenient way to get through firewalls (by using POST for all requests) miss out on the improved performance from HTTP caching, which reduces their performance (and also slightly undermines the basic purpose of a firewall.)

4. Uniform Interface

What really separate REST from other architectural styles is the Uniform Interface enforced by the fourth constraint.

We don’t usually think about it, but it’s pretty amazing that you can use the same Internet browser to read the news, and to do your online banking. Despite these being fundamentally different applications. You don’t even need an extension to the browser to do any of this!

We can do this because the Uniform Interface decouples the interface from the implementation, which makes interactions so simple that it’s easy for somebody familiar with the style to understand it, even automatically (like Googlebot).

The Uniform Interface constraint is made up of 4 sub-constraints:

4.1. Identification of Resources

The REST style is centered around resources. This is unlike SOAP and other RPC styles that are modeled around procedures (or methods).

So what is a resource? A resource is basically anything that can be named. From static picture to a feed with real-time stock prices.

But in enterprise software the resources are usually the entities from the business domain (i.e. customers, orders, products, etc.) On an implementation level, it is often the database tables (with business logic on top) that are exposed as resources. But you can also model a business process or workflow as resource.

Each resource in a RESTful design must be uniquely identifiable via an URI (Uniform Resource Identifier) and the identifier must be stable even when the underlying resource is updated (i.e. “Cool URIs don’t change”).

This means that each resource you want to expose through a RESTful web service must have its own URI. Normally, you would use the first URI below to access a collection of resources (i.e. several customers) and the second URI to access a specific resource inside that collection (i.e. a specific customer):

1) https://api.example.com/customers
2) https://api.example.com/customers/932612

Some well-known APIs that claim to be RESTful fail this sub-constraint. For example, Twitter’s REST APIs uses RPC-like URIs like statuses/destroy/:id and it’s the same with Flickr.

The problem is that they break the Uniform Interface requirement, which adds unnecessary complexity to their APIs.

4.2 Manipulation of Resources through Representations

The second sub-constraint in the Uniform Interface is that resources are manipulated through representations.

This means that the client does not interact directly with the server’s resource. For example, we don’t allow the client to run SQL statements against our database tables.

Instead, the server exposes a representation of the resource’s state. It can sound complicated, but it’s not.

It just means that we show the resource’s data (i.e. state) in a neutral format. This is similar to how the data for a web page can be stored in a database, but is always send to the browser in HTML.

The most common format for RESTful web services is JSON, which is used in the body of the HTTP requests and responses:

{
  "id": 12,
  "firstname": "Han",
  "lastname":"Solo"
}

When a client wants to update the resource, it gets a representation of that resource from the server, updates the representation with the new data, send the updated representation to the server, and ask the server to update its resource so it corresponds with the new representation.

The benefit is that you avoid a strong coupling between the client and server (like with RMI in Java), so you can change the underlying implementation without affecting the clients. It also makes it easier for clients as they don’t need to understand the underlying technology used by each server that they interact with.

4.3 Self-Descriptive Messages

The third constraint in the Uniform Interface is that each message (i.e. request/response) must include enough information for the receiver to understand it in isolation.

Each message must have a media type (for instance, application/json or application/xml) that tells the receiver how the message should be parsed.

HTTP is not formally required for RESTful web services, but if you use the HTTP methods you should follow their formal meaning, so the user won’t rely on out of band information to understand them (i.e. don’t use POST to retrieve data, or GET to save data).

So for the Customer URIs, which we defined earlier, we can expose the following methods for the client to use:

The benefit is that the four HTTP methods are clearly defined, so an API user who knows HTTP, but doesn’t know our system can quickly guess what the service is doing by only looking at the HTTP method and URI path (i.e. if you hide the first column, a person who knows HTTP can guess what it says based on the two new columns).

Another cool thing about self-descriptive message is that (similar to statelessness) you can understand and reason about the message in isolation. You don’t need some out-of-band information to decipher it, which again simplifies things.

4.4 Hypermedia as the Engine of Application State

The fourth and final sub-constraint in the Uniform Interface is called Hypermedia as the Engine of Application State (HATEOAS). It sounds a bit overwhelming, but in reality it’s a simple concept.

A web page is an instance of application state, hypermedia is text with hyperlinks. The hypermedia drives (i.e. engine) the application state. In other words, we click on links to move to new pages (i.e. application states).

So when you are surfing the web, you are using hypermedia as the engine of application state!

So it basically means that we should use links (i.e. hypermedia) to navigate through the application. The opposite would be to take an Customer ID from one service call, and then use it as an input parameter to another service call.

It should work like a good web site where you just enter the URI and then you just follow the links that are provided on the web pages. You don’t need to know more than the initial URI.

For example, inside a customer representation there could be a links section with links to the customer’s orders:

{
  "id":12,
  "firstname":"Han",
  "lastname":"Solo",
  "_links": {
    "self": {
      "href":"https://api.example.com/customers/12"
    },
    "orders": {
      "href":"https://api.example.com/customers/12/orders"
    }
  }
}

The service can also provide the links in the Link HTTP header, and W3C is working on a standard definition for the relation types, so we can use standardized meanings which further helps the user.

An enormous benefit is that the API user doesn’t need to look in the API documentation to see how to find the customer’s orders, so he or she can easily explore while developing without having to refer to out-of-band API documentation.

It also means that the API user doesn’t need to hardcode (and manually construct) the URIs that he or she wants to call. It might sound like a trivial thing, but Craig McClanahan (co-designer of The Sun Cloud API) wrote in an informative blog post that in his experience 90% of client defects were caused by badly constructed URIs.

Roy Fielding didn’t write that much about the hypermedia sub-constraint in his PhD dissertation (due to lack of time), but he later wrote a blog post where he clarified some of the details.

5. Layered System

The fifth constraint is another constraint on top of the Uniform Interface, which says that the client should only know the immediate layer it is communicating with, and not be aware of any layers behind it.

This means that the client doesn’t know if it’s talking with an intermediate, or the actual server. So if we place a proxy or load balancer between the client and server, it wouldn’t affect their communications and we wouldn’t need to update the client or server code.

It also means that we can add security as a layer on top of the web services, and then clearly separate business logic from security logic.

6. Code-On-Demand (optional)

The sixth and final constraint is the only optional constraint in the REST style.

Code-On-Demand means that a server can extend the functionality of a client on runtime, by sending code to it that it should execute (like Java Applets, or JavaScript).

I have not heard of any RESTful web services that actually send code from the server to the client (after deployment) and gets it executed on the client, but could be a powerful way to beef up the client.

A really nice feature of the simplicity that is enforced by these six constraints (especially, uniform interface and stateless interactions) is that the client code becomes really easy to write.

Most modern web framework can figure out what to do, if we follow the conventions above and they can take care of most of the boilerplate code for us.

For example, in the new Oracle JET toolkit, we simply need the JavaScript below to create a customer (and it would be just as easy in AngularJS):

// Only code needed to configure the RESTful Web Service
var Customer = oj.Model.extend({
  urlRoot: "http://api.example.com/customers",
  idAttribute: "id"
});

// Create a new customer representation
var customer = new Customer();
customer.attributes.firstName = "Han";
customer.attributes.lastName  = "Solo";

// Ask the server to save it
customer.save().then(function() {
  console.log("Saved!");
});

And it’s just as easy to call the other HTTP methods.

So the front-end engineer just need to add a few more lines to add an HTML form where the user can enter the values, and voila we have a basic web app!

And if we use one of the many gorgeous UI frameworks (like Twitter’s Bootstrap or Google’s Materialize), we can quickly develop something really nice looking in a really short time.

That’s it for now. Thank you for reading and take good care until next time!

That’s nice, you might think, but why should I use it?

Because it will allow you to show off against other API Designers by claiming that your REST services are twice as RESTful as theirs

But more seriously, Roy Fielding, who invented the REST architectural style, didn’t add caching as a requirement just for the fun it! He added it because it can seriously boost performance, which is also shown in the numbers in Tom Christie’s great post on performance tuning of Django REST services.

So, how do you get started with HTTP caching?

It’s only for HTTP GET requests!

At first it may seem like an overwhelming task to implement HTTP Caching; especially if you have already developed a huge number of services.

The good news is that it’s only GET methods where you need to think about caching as it doesn’t really make much sense to cache POST, PUT or DELETE responses.

The even better news is that if you simply specify the right HTTP header then the browser will do all the heavy lifting for you!

Code, please!

That’s all very nice! But can you please show us the code?

Definitely, but the only code you need is the Cache-Control header in your HTTP response. There are a number of directives in this header you can use to control the caching:

Cache-Control: max-age=3600

Cache-Control: max-age=0, s-max-age=3600

Cache-Control: private, max-age=3600

Cache-Control: public, max-age=3600

Cache-Control: no-store

Cache-Control: no-cache

Cache-Control: max-age=3600, must-revalidate

Cache-Control: s-max-age=3600, proxy-revalidate

So let’s say that the client sends a request for some metadata, and we want the client to cache it for 1 hour:

GET /customers/metadata HTTP/1.1
Host: api.example.com
Accept: application/json
Accept-Language: en

To do this, we just add the Cache-Control header to our response:

HTTP/1.1 200 OK
Content-Type: application/json

Cache-Control: max-age=3600

Content-Length: 88
Etag: "6d82cbb050ddc7fa9cbb659014546e59"

{
  "languageCodes": [
    {"da":"Danish"},
    {"no":"Norwegian"},
    {"en":"English"}
  ]
}

As you can see it’s pretty easy to add caching to your RESTful services…

So if you have performance issues then HTTP caching could be the power tool you are looking for to seriously reduce your response times!

Especially if it has occurred silently over a long period of time, so when it’s discovered it’s too late to rollback to a backup before the defect was introduced. It’s even worse if it has also occurred randomly, so there is no pattern to base your fix upon.

Yet an awful lot of REST APIs ignore concurrency control, so if they are used by multiple clients who modify the same data at the same time then it can lead to lost updates and stalled deletes, which slowly ruins the data in the database.

This is totally unacceptable in most enterprise applications where data integrity is something you just don’t fool around with…

So how can you avoid this messed up situation?

You use the concurrency control designed into the HTTP protocol as a simple, yet effective way to protect the integrity of your data.

Meet the ETag Header

If you want to use the concurrency control in the HTTP Protocol, you need to use the optional Entity Tag (ETag) header in the HTTP request.

The ETag is kind of like a version stamp for a resource and it’s returned as part of the HTTP response.

For example, if you send a request to get a specific customer:

GET /customers/987123 HTTP/1.1

Then the ETag (if used) is included in the header in the response:

HTTP/1.1 200 OK
Date: Sat, 30 Jan 2016 09:38:34 GMT

ETag: “1234”

Content-Type: application/json

{
  "id":987123,
  "firstname":"Han",
  "lastname":"Solo",
  "job":"Smugler"
}

Each time the resource is updated on the server, the ETag header will be changed to reflect the content of the new version of the resource.

So to avoid lost updates, you simply take the value of the ETag header and put into the If-Match header on the PUT request:

PUT /customers/987123 HTTP/1.1

If-Match: “1234”

Content-Type: application/json

{
  "id":987123,
  "firstname":"Han",
  "lastname":"Solo",

“job”:”General”

}

The PUT request above says that you want to update the customer resource on the server, but only if the ETag matches 1234 to make sure that the customer hasn’t been updated since you sent the GET request. In this way, your request won’t incidentally overwrite other users’ updates.

Over at the Server

When the server gets the PUT request, it will execute the logic below:

First, the server checks that the If-Match header is included in the request. If not, it will tell the client that you cannot update this resource without the If-Match header.

Second, the server checks if the resource actually exists as it must exists before it can be updated.

Third, it checks if the ETag supplied in the If-Match header is the same as the latest ETag on the resource. If not, it tells the client that the precondition has failed.

Finally, if the request passes the three validations, then the server updates its resource.

If the server uses the same approach on DELETE request, you can also avoid stalled deletes.

Note: The logic above assumes that the server doesn’t allow you to create new resources with a PUT request. If this is allowed then the first step should be to check if the resource exists, and if not then branch out and create it and skip the other steps.

Implementation Hints

There are several ways to implement ETags on the server.

One way is to make a hash of the resource, and put in the ETag header. But you need to make sure that the hash includes all updatable fields in the response, be sure that there cannot be hash collisions, and find a hashing algorithm that doesn’t impact performance too much.

Another really simple way to implement ETags is to add a read-only etag column to the underlying database table and add a trigger that increases the value each time the row is updated. Of course, the server needs to be aware that the database changes this value behind the screen, so the server always uses the latest version.

Why not timestamps instead of ETags?

An easier way to implement concurrency control in HTTP is to use the Last-Modified and If-Unmodified-Since headers instead of ETag and If-Match. This difference is simply that these two headers use timestamps instead of ETags.

So, if it’s easier why not use it?

The problem is that the timestamps use seconds as their finest precision, so if you have fast, high-frequency updates then there is a risk that two updates occur within the same second and you lose one of them.

So in enterprise software where data integrity is an absolute requirement, Etags are the safe choice.

The problem is that one of the foundations of the REST architecture is that the client-facing representation of a resource must be independent of the underlying implementation, and implementations details should definitely not be leaked to the client, which is all too easy with the database-driven approach.

It’s also important to ask yourself if an almost raw database is the best interface you can offer your API users? I mean there is already a near-perfect language for doing CRUD operations on database tables, it’s called SQL… And you probably have some business logic on top of those tables that your API users would appreciate not having to re-implement in their own code.

So how do you move beyond this database-oriented thinking and closer to a more RESTful design for your API?

Let’sThe mapping between SQL and HTTP is deceptively easy. find out…

1. Begin with the API User in Mind

Bestselling author and architect Sam Newman’s great book on microservices provides a powerful alternative to the database-driven approach for designing REST web services. It’s useful even if you don’t plan to use microservices.

Newman suggests that you divide your application into bounded contexts (similar to business areas). Each bounded context should provide an explicit interface for those who wish to interact with it. Implementation details of the bounded context that don’t need to be exposed to the outside world are hidden behind the interface.

You should use this explicit interface as the basis for your API design. Start by asking yourself what business capabilities do the API user needs, rather than what data that should be shared. In other words, ask yourself what does this bounded context do? and then ask yourself what data does it need to do that?

The promise is that if you wait with thinking about shared data until you know what business capabilities you need to offer, it will lead to a less database-oriented design.

I think his approach is a good way to jolt you out of the database-driven mindset, but you need to be careful that you don’t end up designing a REST-RPC hybrid.

What I also like about this approach is that it minimizes the interface and doesn’t expose all data by default, but hides internal data (like logging and configuration tables) from the client and instead focuses on what the client actually needs.

This also fits beautifully with veteran API designer Joshua Bloch’s maxim saying that When in doubt, leave it out (from his highly popular presentation on API design), and it also harmonizes with the REST principle that a representation of a resource doesn’t need to look like the underlying resource, but can be changed to make it easier for the client.

So feel free to think about what would be the easiest interface for the API user, and then let your resource take data from multiple tables and leave out columns that are irrelevant to the job that clients need to perform.

2. Use Subresources to Show Relationships

An attractive alternative to only using top-level resources is to use subresources to make the relationships between resources more obvious to the API user, and to reduce dependencies on keys inside the resource representation.

So how do you decide what resources should be subresources? A rule of thumb is that if the resource is a part of another resource then it should be a subresource (i.e. composition).

For example, if you have a customer, an order and an order line then an order line is a part of an order, but an order is not a part of a customer (i.e. the two exists independently and a customer is not made up of orders!)

So the URIs would look like this:

/customers/{id}

/orders/{id}/lines/{id}

A different rule of thumb is to also include aggregations as subresources. That is, a belongs to relationship. If we use this rule then an order belongs to a customer, so the path would look like this:

/customers/{id}/orders/{id}/lines/{id}

So what rule should you pick?

The idea with subresource is to make your API more readable. For example, even if you don’t know the API you can quickly guess that POST /customer/123/orders will create a new order for customer 123.

However, if you end up with more than about two levels then the URI starts to become really long and the readability is reduced.

You also need to be aware that subresources cannot be used outside the scope of their parent resource. In the second example, you need a customer id before you can lookup a order, so if you want a list of all open orders (regardless of customer) then you cannot do it in the second example.

Ehh, so what to pick?

If you want a flexible API, aim for fewer subresources. If you want a more readable API, aim for more subresources.

The important thing is that whatever rule of thumb you pick then be consistent about it. I mean the API user might disagree with your decision, but if you are using it consistently throughout your API, he or she will probably forgive you.

3. Use Snapshots for Dashboard Data

A deal-breaker for using subresources is that the client might need to access data across subresources to get data for a dashboard or something similar. For example, a manager might want to get some statistics about orders across all customers.

Before you go ahead and flatten your whole API, there are two alternatives you should consider.

First, remember that there is nothing that prevents you from having multiple URIs that point to the same underlying resource, so beside /customers/{id}/orders/{id}, you could add an extra URI to query orders outside of the customer scope:

/orders/{id}

To minimize the duplication of functionality, you can limit the top-level /orders URI to only accept GET requests, so if clients want to create a new order, they will always do it in the context of a customer.

Of course, you need to be careful not to duplicate resources unnecessarily, but if there is a customer need, then there is a customer need, and we need to find the best possible solution.

In RESTful Web Services Cookbook, Chief Engineer at eBay (and former Yahoo Architect) Subbu Allamaraju suggests an alternative approach called snapshots.

For example, if an order manager wants to see some specific statistics (5 latest orders, 5 biggest clients, etc.) then we can create a snapshot resource that finds and returns all these information:

/orders/snapshot

I personally like the snapshot approach better, because it doesn’t feel like querying a database. But with that said, the snapshot approach requires an intimate knowledge of the API user, and the extra order top-level resource will offer more flexibility.

4. Use Links for Relationships

Another way to show relationships between resources, without falling back on using keys in an SQL-like manner, is to embed links inside your responses:

{
  "id": 123,
  "title": "Mr.",
  "firstname": "Han",
  "surname": "Solo",
  "emailPromotion": "No",
  "_links": {
    "self": {
      "href": "https://api.example.com/v1/customers/123"
    }, 
    "contactDetails": {
      "href": "https://api.example.com/v1/customers/123/contactDetails"
    },
    "orders": {
      "href": "https://api.example.com/v1/orders?customerId=123"
    }
  }
}

A cool thing about links is that they allow autodiscovery by clients! When the client gets the response back then it can see in the _link section what other actions it can follow from here. This is just like when you are surfing the web where you come to a page and then you can follow its links to new pages.

Another nice thing is that clients will have fewer hard-coded links in their code, which will make the code more robust. If the client wants to see the customer’s orders, it can just follow the orders link to get them.

However, there are different opinions about if you should use links, or not…

Vinay Sahni writes in his excellent blog post Best Practices for Designing a Pragmatic RESTful API that links are a good idea, but we are not ready to use them yet. On the other hand, the RESTful maturity model says that when you start using links you have reached the highest level of REST maturity.

So what to do?

Well, Dr. Roy Fielding, an expert on software architectures and the inventor of REST architectural style, flatly said on his blog that if you don’t use links it ain’t REST services, and he kindly encourages you to use another buzz word for your API!

5. Hide Internal Codes

In an earlier post, I incidentally leaked internal codes in the job_id column on the employees table:

{
  "jobId": "SH_CLERK"
}

Needless to say, this is an implementation detail that gives away that we are using a relational database, and experienced Oracle users would instantly spot that it’s Oracle’s sample HR schema. This leaking makes it harder to switch to a document-oriented database, like MongoDB, where there is no concept of foreign keys.

But even if the chance of switching to MongoDB is zero, it still makes the response harder to read.

So a better approach is to let the REST API translate the internal code to the human-readable value that the code represents (i.e. “Shipping Clerk”) and then also remove the Id part of the field name.

{
  "job": "Shipping Clerk"
}

This version is definitely more readable, but a fair concern is if the service will be slower now that it needs to lookup the value? I used to be an avid reader of Tom Kyte, the Oracle DB expert, and still remember that you should always optimize from measurements. I mean there’s a good chance that the HTTP cache will help us out and make it less of a bottleneck than it appears at first glance.

As a rule of thumb, if performance means everything to you (or you have a lot of lookup fields) then you might consider leaking the internal codes. Otherwise, you should provide a more readable API by hiding them.

6. Translate Automatically

But what about translations? What if you have a multilingual application that has translations of the internal code in multiple languages? How do you handle that?

Simple! You let the API user specify its preferred language in the Accept-Language HTTP header. For example, Accept-Language: da and then the REST API should automatically translate it into Danish:

{
  "job": "Shippingmedarbejder"
}

7. Create a Resource for Metadata

So what if the API user needs to show a drop-down list that shows all possible jobs? How will he or she get a complete list of all possible values for the job field?

The easy solution for you is simply to write the list of possible values in your API documentation and then the API user can hardcode them in the drop-down list. But this will lead to fragile client apps that need to be updated when new job types are added, and it just doesn’t feel very web-like to rely on offline metadata.

A more robust solution is to create a metadata subresource that provides list of values and other metadata that are needed when using the resource. For example, a /employees/metadata subresource could provide the API user will all the metadata needed to interact with the employees resource.

This solution is similar to how Atlassian is doing in the JIRA API. If you make sure that the response from the metadata subresource is cached properly it shouldn’t affect performance adversely and you will provide a more flexible API that leads to more stable client apps.

That’s it for now. I really hope that some of these tips will help you design better REST APIs. Thanks for reading and take good care until next time!

You can even create an attractive niche brand by combining your technical expertise with a deep understanding of how your industry works.

That’s nice, you might think, but how do you get this industry knowledge?

Here are six tips to get you started:

1. Create a mental model: Make a boxes and arrows diagram of your current mental model of the domain. The remaining tips are all about growing and refining your model.
2. Google it: A simple Google search, like “domain knowledge banking”, will give you a lot of good reads to get started with.
3. Find a mentor: Find someone who really understands the domain and listen curiously to what they say about their work. People usually enjoy talking about what they are good at.
4. Follow the experts: Find out who are the thought leaders of the industry, and follow their Twitter accounts and blogs.
5. Read the trade journal: Find out what the major publication is in the industry and read it.
6. Watch an employee at work: See somebody perform one of the key jobs in that industry. For example, spending a few hours with a call center employee will give you a whole new understanding of their fast-paced job.

The good news is that the fundamentals of most industries remain pretty stable over time, so once you have a basic understanding it won’t require a lot of maintenance.

And since most software engineers don’t bother to learn much about the industries they work in, you can easily get ahead with only a small investment of your time.

In each 90-minute block you focus intensely on a given task, so no social media, chatting, emailing, news reading, etc. You just work, work, work, and then work some more on the task at hand.

Some people has experienced that they can get as much done in a focused 90-minute block as they normally get done in a whole day!

Research shows that 90 minutes is the upper limit for how long we can stay focused. After that our minds start wandering and become less productive.

This is why you need a break after you have worked intensely for 90 minutes. It needs to be a real break away from the computer, so your brain can recover and become ready to performance at its maximum level again.

So how does such a 90-minute schedule looks in practice?

For a blogger it can look like this:

• 08:00-09:30 Block 1: Bulldoze through first draft.
• 09:30-10:00 Break: Go for a run.
• 10:00-11:30 Block 2: Rewrite and make ready for publication.
• 11:30-12:00 Break: Enjoy lunch.
• 12:00-13:30 Block 3: Publish and promote.

While 3×90 minutes doesn’t sound like a lot, you must remember that this is highly effective work without any procrastination. Bestselling author Tony Schwartz wrote in Harvard Business Review that he used to write for 12 hours each day and finish a book in about a year. He then switches to write in three 90-minute blocks per day and was able to finish a book in less than 6 months!

But if you really want to push it, personal development expert Steve Pavlina puts down the challenge of doing 5×90-minute blocks per day to get a whole week’s work done in a single day!

For me it works best if I have a goal for each 90-minute block. It’s not needed with a specific and measurable goal, but just a rough idea of what I would like to achieve within the 90 minutes.

I also discovered that I need a physical kitchen clock with a countdown feature to make this work for me. At first I used the timer on my iPad, but after some time the screen goes dark, and I must be able to see the clock counting down all the time for maximum motivation.

My secret weapon, a kitchen timer!

Another essential element for success is to take breaks seriously(!) So don’t sit down and “do” Facebook for half an hour. Take a real break and give your mind a chance to relax and renew itself.

Here’s a quick list of ideas for break activities:

• Take a run, stretch or some other exercise.
• Go for a walk.
• Meditate, pray or visualize.
• Enjoy a healthy snack.
• Take a nap.
• Talk with a friend.

If you work in a management position (or another job with many interruptions) it can be difficult to organize your work into uninterrupted 90-minute blocks. But you should still carve out one 90-minute block per day to get the important, but not (yet!) urgent, stuff done. Everybody can do that!

In this way, you can learn about potential issues before they turn into serious problems.

Your managerial reward of effective one-on-ones is a lack of drama in your team, which enables your people to spend their time on doing great work!

So how do you do effective one-on-ones? Here are 9 tips to get you started.

1. Be consistent and never-ever cancel it

You should schedule one-on-one meetings as a reoccurring event in your calendar. The meeting should be 30-60 minutes. You should schedule it for every week, but if you have more than 6 employees then consider scheduling it biweekly due to the time involved.

You should never-ever cancel it, as that will basically communicate that your people are not that important to you.

If you keep it consistent your people will know that this is the time to discuss issues. This will make your schedule more predictable as you are less likely to be interrupted at other times if your people know that there’s a one-on-one where they can bring up their concerns.

2. Start with “How are you?”

Michael Lopp, a veteran software engineering manager, writes in The Update, The Vent, The Disaster that you start the one-on-one meeting with a simple, “How are you?”

It’s a soft opener, but the idea is that whatever the reply, you can learn something useful and take the conversation from there.

3. Listen, Listen and Listen Some More

The one-on-one should be a bottom-up meeting where your people share information with you, rather than a top-down where you are pushing information down their throats!

While most managers are extremely busy, and honestly don’t want to hear about more problems, which will only make them more busy and perhaps even shake the boat! But the hard truth is that the earlier a problem is detected, analyzed and dealt with, the cheaper it will be to solve. And all practical experience shows that ignoring problems will (unfortunately!) not make them disappear.

4. It Ain’t a Status update!

The one-on-one is not for operational stuff. There are project meetings and stand-up meetings for handling the daily status of tasks.

One-on-ones are for strategic stuff; like, looking into how we are doing stuff and how we can do it better.

If your employee starts to go through status, interrupt after some 5 minutes and say, ”It sounds like you got the project/task under control, but how about the [insert strategic activity] that we talked about earlier?”

5. Ask Powerful Questions

Sometimes it can be difficult to get the conversations rolling. For example, when you have new people, or very quiet people, or people who have had a poor manager in the past.

So to get the talking started it always help to have some powerful questions prepared. For example:

1. What excites you at work?
2. Do you have ideas for how we can do things better?
3. Where is your job satisfaction on a scale 1-10? Follow-up with: What would it take to make it a 10?
4. Do you have ideas for the next release of our product?
5. How’s the family?

A one-on-one is also an excellent opportunity to debrief recent work. For example, to discuss what went well, what didn’t went well, and what are the lessons learned for next time.

6. Ask for Gossip (really!)

It’s always good to ask for gossip. Especially, if there has been a recent re-org or any other major event.

Ask your people about their opinion about the event, rather than starting with your own opinion. Then, afterwards, you can always come with your interpretation of the event and explain how it fits into the big picture.

7. Follow-up (action speaks louder than words!)

If there are any actions coming out of the meeting, be sure to follow-up and execute them.

If actions are agreed and nothing ever happens with them, it will destroy trust in the relationship.

Be the manager who shows that good ideas are acted upon. Show with your actions that what’s being said in a one-on-one is important to you. Build trustworthiness through your actions.

8. A Dedicated Day for One-on-Ones

Software development manager and blogger Ed Gibbs writes in One-on-Ones in a Single Day that he prefers to have all his one-on-ones in a single day (Wednesday).

The benefits are that it’s easier to defend a full day (than large chunks of time throughout the week), it frees large chunks of time on the other days for other activities, you are better prepared for the one-on-ones, and it gives you a better sense of how the whole team is doing.

9. Try a Different Location

Executive coach Beth Miller recommends in Revamp Your Approach to Monthly One-to-One Meetings that you sometimes move away from the office to shake up the one-on-one, add a more casual theme to it and get to know your people on a different level.

For example, by going out for lunch or by going for a walk outside, which was a favorite of Steve Jobs.

At the end of the day, a one-on-one is an opportunity for your people to share their ideas, concerns and need for personal development. Your task as a manager is to ask good questions, listen to what is being said, provide sensible feedback, and follow-up constructively.

Some managers will undoubtedly read this and think that it sounds like a humongous time stealer from another dimension! But they must remember that their job is management and that one-on-ones are one of the most effective management tools out there.

In the words of Johanna Rothman, a bestselling author of software management books, “Managers who use one-on-one meetings consistently find them one of the most effective and productive uses of their management time.”

Churchill was born into an English aristocratic family with all the expensive habits of this class, but unfortunately his late father had failed to leave him a family fortune that is needed to sustain such a lifestyle. A reasonable man would have reduced his expenses to meet his actual income — but Churchill never thought small — so he decided to simply raise his income until it met his expenses and if his expenses outgrew his income again, he would simply grow his income even more.

Now the curious reader will probably agree with Churchill’s logic in theory, but may ask if an English aristocrat in the 20th century was educated with lots of market-relevant skills that could actually produce an income? And on top of that a substantial income that could support an extravagance lifestyle with French champagne, Cuban cigars, world travels, country house, servants and so forth?

To start from the beginning, when Churchill was a young officer in the British Army, he was desperate to seek out action as courage in battle was essential for promotion. So when a military expedition started in Northwestern India (today, Pakistan) to punish the local tribes, he wanted to join despite the travel expenses were more than what his mother’s endowment could support. So he made an agreement with The Daily Telegraph, a newspaper, that he would write articles about the expedition to the newspaper in exchange for them paying his travel expenses.

The series of articles showed that he had a talent for words and after the expedition had ended, he turned the articles into his first book, The Story of the Malakand Field Force, which is still a highly readable book about his baptism of fire, which he describes as a delightful experience: “Nothing in life is so exhilarating as to be shot at without result.”

Churchill quickly realized that his talent for words could be his road to financial freedom, so he continued to hone his writing skills and moved from journalism to more substantial literary works. He was never afraid of demanding a high fee for his writings and he refused to give in to anyone who wanted a low price. At some point the frugal Rockefeller family approached him to write the biography about the late John D. Rockefeller, supposedly the richest man in history, but the Rockefellers withdrew their offer in horror once they saw the fee that Churchill demanded for writing the biography. So Churchill never wrote the Rockefeller biography, but instead continued with other works supported by more generous sponsors.

His Writing Habits

Obviously you can only demand unreasonable high fees, if your writings are of such an outstanding quality that people still think they get value for their money. And at the same time you need to produce them at a rapid pace, so your income stream doesn’t run dry. So how did Churchill manage to produce great works at a high speed while still having time for a political career?

Churchill preferred writing late in the evening and usually started around 10 PM when dinner ended and continued writing until 2 AM. In these late-night writing sessions Churchill would be standing by his upright desk with a whisky soda in one hand and a cigar in the other while using his formidable oratory skills to turn historical events — for he wrote mostly about history and politics — into a captivating narrative that his sectary would dutifully write down. His research assistant would also be there ready to supply facts as needed and spot inaccuracies in the narrative.

As Churchill sharpened his writing skills over the years, he also developed a set of habits that he followed to produce his successful works:

• Give the reader a good ride: When Churchill wrote Marlborough: His Life and Times he said to his research assistance, “I aim to give the reader a good ride”. This focus on telling a great story, rather than being overly analytical, is probably why his books are still a good read for the casual reader.
• Dictate, rather than write: Churchill dictated his words to his sectary, rather than writing them himself. The reason for this might be that speaking is the natural medium for a great orator, such a Churchill, so it gives the narrative a more natural flow. Another reason is that it’s faster to speak than to write, so he could simply get more done in less time by dictating.
• Use short words: Churchill said, “Short words are best and the old words when short are best of all.” Churchill knew that short English words (which are often derived from Anglo-Saxon, rather than Latin) have a more powerful effect on the reader. As noted in this style guide, when Churchill asked for material help from America in the fight against Hitler, he didn’t say “Deliver to us the implements, and we will complete the assignment”, instead he simply said, “Give us the tools, and we will finish the job.“
• Seek feedback: When working on a manuscript Churchill was relentless in seeking feedback from others. For instance, when writing the Marlborough book he sent 308 letters asking for answers to specific questions, or for the recipient to review his manuscript.
• See it in print before it’s final: Churchill would always get his draft manuscript set up by a typographer and printed, so he could see what the final product might look like while he still had a chance to edit it — and this was before the computer, so it was a cumbersome process. Even if his publisher insisted that this was an unnecessary step in the creative process, Churchill would offer to pay for this out of his own pocket.
• Take a siesta: When Churchill was a young war correspondence in Cuba, he picked up two lifetime habits — the first being smoking Cuban cigars and the second being taking a siesta. At around 5 PM each day, he would sleep for 1½ hour and this was no power nap where he just sat in a chair. No, he would get his clothes off and go to a proper bed and sleep. Afterwards, he would take a bath and get dressed for dinner. Churchill said that taking a nap allowed him to work 1½ day in a single day and the nap was probably also needed to have the energy for his late-night writing sessions.
• Set a daily aim: While writing his biography on Marlborough and building a cottage, at the same time, Churchill had the aim of laying 200 bricks and writing 2,000 words per day. He knew if he managed to do both then it had been a good day.

Churchill followed these writing habits throughout most of his career and was a highly productive author who wrote 15 books — some of which were multi-volume works consisting of more than 4,000 pages — together with a vast amount of articles, speeches and letters. You can almost be surprised that he also had time for a political career…

His Finest Hour

When Churchill was chosen as Prime Minister in 1940 to lead England in the war against Hitler, he wrote in his diary, “I felt as I was walking with destiny, and that all my past life had been but a preparation for this hour and for this trial.”

While his military and political careers had undoubtedly prepared him for this grand challenge, his mastery of words was an equally indispensible tool when he needed to strengthen the moral of the British people, which was seriously needed as Hitler continued to move from one aggressive triumph to another.

And Churchill did indeed mobilize his formidable writing skills in the defense of England. In less than two weeks, he wrote three speeches Blood, toil, tears and sweat, We shall fight on the beaches and This was their finest hour which are all are among the greatest speeches delivered in the history of mankind.

And England did not back down, but stayed in the fight until Russia and America also joined the war effort and together they defeated Hitler and won the war.

In 1953, his literary career was crowned with a Nobel Prize in Literature. When reading his books, or listening to his speeches during the war, you tend to agree with the Nobel committee that awarded him the prize for “his mastery of historical and biographical description as well as for brilliant oratory in defending exalted human values.“

Ian Fleming didn’t start writing until he was in his forties. Before his novelist occupation he had an admirable career in the British Naval Intelligence. During World War II he planned “Operation Goldeneye”, which was the contingency plan for how the British would defend Gibraltar, if the Germans had attacked it through Spain. He was also involved in “Operation Mincemeat”, which was a deception manoeuvre that misled the Germans into thinking that the Allied forces in Africa would invade Southern Europe through Greece, rather than Sicily where it actually happened. And after the D-Day, he helped selecting targets for T-Force, which was a special military unit that captured high-value targets (e.g. individual scientists, nuclear research, rocket technology) before the Russians got them.

After the war he stopped in the Naval Intelligence with the rank of Commander and decided that his new objective in life would be the pursuit of pleasure. He secured a job as Foreign Manager at Kemsley Newspapers where his contract included 8 weeks of holiday per year, so that he could spend January and February in the Caribbean, rather than live through the cold winter in London.

He bought a piece of land on Jamaica (then, a British Colony) with a private beach and reef, and got a local contractor to build a simple house with a great view of the Caribbean Sea. He christened the house Goldeneye.

Goldeneye would always be the destination for Fleming’s long winter holiday, and he used these winter months in Jamaica for sunbathing on the beach, sailing out to the reef to swim with a mask and a spear, catch lobsters for dinner and entertain visitors from near and far.

This easy and pleasurable life lasted rather uninterruptedly till 1952. Ian had been the most professional bachelor and had never established any deep emotional connections with any of his many girlfriends. However, at the age of forty-three he had finally ended up marrying his long-term mistress, Lady Anne Rothermere (later, Ann Fleming) as he had made her pregnant and her husband therefore wanted to divorce her.

That winter due to the pressure caused by the pending divorce, there weren’t many visitors to Goldeneye. There were many different worries that haunted Fleming. Would he manage to take care of a family at this age? Would it change his life? Would he be able to provide Ann with the same stability as her former and much wealthier husband, Lord Rothermere? All those questions kept circling around in his head, but remained unanswered…

Ann kept herself occupied by sitting in the garden wearing her large straw hat and painting flowers. Ian didn’t find any pleasure in painting, so Ann suggested that he should start writing something to distract his thoughts.

So to “take my mind of the shock at getting married at the age of forty-three” as he later said, he decided to write the spy novel to end all spy novels, and gave it the title Casino Royale.

Ian was a man with a boyish enthusiasm, so when he started such a project, he poured all his energy into it. He was also a man of habits, so he integrated writing into his daily routine, which he would follow day in and day out without exceptions.

His Daily Routine

He started his day with a morning swim in the Caribbean Sea followed by a breakfast with Ann. The breakfast always consisted of scrambled eggs, bacon and black coffee as he insisted that tea tasted like mud and was the reason behind the decline of the British Empire.

At 9am, he would give Ann a kiss, leave the breakfast table and go inside into the main living room in Goldeneye. He would close the jalousied windows to create a cool and shady room with a hint of a tropical breeze. Then he would take out his old Imperial portable typewriter and type for the next three hours.

He used his wartime experiences as a starting point for activating his imagination and created the world of James Bond. He would write non-stop and wouldn’t worry about what he would put in as he could always edit it later.

At noon, he would stop writing and go outside in the warm Jamaican sun. Together with Ann he would go down to the beach to sunbathe and swim before lunch. After lunch, he would sleep for an hour or two.

Around 5pm, he would be ready to continue his work. He would go into the main living room again, and use an hour or so to read through what he had written during the day and make the necessary corrections. After he finished the corrections, he would take the papers and place them in a drawer and thereby end his workday. And by 6:30, he was ready for his first drink of the day and to enjoy the evening without worrying about his work.

Six weeks after Ian had started writing for the first time, he completed Casino Royale. The novel was a tour through a world of gambling, violence, betrayal, torture and sex. By the end of the tour, Le Chiffre (the bad guy) had been annihilated, and so had Vesper Lynd (the girl), while Bond had completed his first mission.

And for the next many years, Ian would continue to follow this elaborate, yet simple, writing schedule and complete a new Bond novel at Goldeneye each winter.

Why Did He Follow a Routine?

His writing habits raise some interesting questions. The first is why would a man of creativity and imagination, like Ian Fleming, follow such a strict regime, rather than just write whenever he felt like it? Obviously it served as a reliable safeguard against abandoning a project when feeling moody or unmotivated. He would later advise another writer that you should always follow your routine, because that will keep you going even when you feel that your writing is nonsense and nobody will ever read it.

Another interesting question is how he managed to be so productive when only putting in 4 hours of actual work per day? The reason must be that he was much more effective during those hours than an average person. He had the luxury of not being interrupted and did not engage in any activities that were not related to the task at hand. He completely focused on the single task of writing this particular novel what probably helped him enter a flow mode where he was extremely productive.

Another reason could be that writing was his true vocation in life, which is why once started, it went so effortlessly that he only needed to spend only a few hours per day and yet remain efficient. He was undoubtedly gifted with an extraordinary imagination and ability to generate ideas (which had also served him good in his previous occupation), but he was also able to create an environment where these talents were expressed most effectively. Fleming certainly had talents, but he was also able to use them to their outmost when writing his novels, which was for the overall benefit — of his own and of his grateful readers.

There are some important lessons to be learned from Ian Fleming’s writing habits. The first is that a single-minded focus on a specific task for three hours in a row without any risk of interruptions enables you to enter a flow mode where you can enjoy a much higher productivity and creativity. The second is that forcing yourself to take daily actions, regardless of the mood, will eventually lead to success. But was Fleming successful? I would say so. His Bond novels have sold more than 100,000,000 copies, which seems like a fair amount for a project that just aimed at distracting him from the fact that he was getting married a little late.

His mastery of the word was gained over a lifetime as a journalist and author where he gradually learned the do’s and don’ts of writing. In this post I’d like to share three of the best tips he ever gave to other authors about the secrets of great writing.

1. Write in the Morning

Hemingway was a firm believer in writing in the morning — from around 9am till 2pm — because he could start with a fresh mind after a night of sleeping. He knew that a clear mind was essential, so he resisted reading anything before he started writing. He believed that reading other people’s words would fill his mind with their ideas and drown his own true voice.

He also learned that the morning was the part of the day where he was most likely to avoid interruptions. Similarly, he knew that the thought of writing in a café was seductive to most authors, but the harsh reality was that authors need tranquillity to write. His own experience proved that sitting alone in his hotel room in the morning he was much more likely to produce great literature than in a busy, noisy café with endless possibilities of being interrupted.

2. Don’t Empty the Well

Hemingway knew the importance of breaks in creative work. He was careful never to empty the well, which means keep writing until you are completely drained. Rather, he would stop writing while he knew exactly what to write next.

His experience was that if he let a little water stay in the well (i.e., not drained himself of ideas), it would automatically replenish during the non-writing part of his day and his mind would be full of creativity the next morning when he would start writing again.

He also firmly resisted talking or thinking about his writing during the non-writing part of his day, because the unconscious mind should have peace to replenish the well without being interrupted by the conscious mind. So if he started thinking of his story, he would immediately force himself to think of something else.

3. Be Concise

Hemingway started his career as a newspaper reporter when telegrams were charged by the number of words in them. In this way he learned early to keep his writing concise and use short declarative sentences.

He wrote only about 400-600 words per day — in comparison this post is 520 words — but was careful to select the right words. Some were surprised about how few words he produced per day and questioned his tight style. So to prove his point Hemingway made a bet with a few other writers whether he could write a story in just six words. He won the bet by writing on a napkin: For sale: Baby shoes, never worn.