Unlocking Success: 5 Challenges in Building Open Source SDKs and How to Conquer Them

Creating open-source libraries and SDKs is a distinct endeavor compared to developing applications. It’s an adventure filled with numerous obstacles that demand expertise and experience to navigate effectively.

As the Frontend SDK lead at Frontegg, our mission is to create an open-source library for easy integration of Frontegg components into developers’ applications. Our SDKs include an embedded login box and a robust admin portal, offering extensive user management capabilities on top of our frontend APIs. We provide dedicated SDKs for various frameworks, such as frontegg/react, which is widely adopted by both B2B and B2C frontend developers with 10,000 weekly downloads.

In this blog, I will shed light on the challenges that resonate with my daily experience, challenges that I believe every SDK developer can relate to. Along the way, I’ll provide helpful tips and illustrative examples. Let’s jump in:

Challenge #1. Backward Compatibility vs. Breaking Changes

Creating an excellent SDK is like juggling. Youn need to keep everything working as before while making constant improvements.

Now, let’s be honest, developers have tight schedules. Asking them to embark on an adventure to upgrade your libraries and actively rework their code is not a great idea. What they truly appreciate is seamless delivery of bug fixes, new features, and slick improvements  This approach, while developer-friendly, poses a unique set of challenges for us.

1. Strive for the best while keeping the old
   SDK developers are committed to enhancing the user experience by identifying opportunities for codebase improvements, which may include critical changes, or restructuring efforts to optimize the SDK. When such improvements are recognized, developers face the dilemma of either compromising on a single solution or introducing a new one while retaining support for the previous version.
   
2. Avoid code proliferation
   To address the initial challenge, we sometimes introduce a new configuration option that accommodates both the new and old systems, ensuring backward compatibility. However, managing multiple code versions can be impractical for several reasons:
   

• Dev velocity
  Slows down development, as developers must handle multiple codebases.
  
• Code quality
  The codebase becomes convoluted and challenging to maintain, leading to spaghetti code.
  
• Expertise
  Over time, maintaining legacy code interferes with developers’ expertise in the product.
  

3. Dual thinking—backward and forward
   Being an SDK developer requires thinking both backwards and forward. Achieving 100% backward compatibility upon release is essential, but developers must also anticipate how their work today can avoid future breaking changes.

Balancing these factors and managing a single, coherent codebase is crucial for the success of open-source SDK projects. Lets jump to some practical tips to achieve this:

Practical Tips

1.Keep it generic and object-oriented

No! (Do not do this)

Try this instead.

2. Think ten features ahead
Always envision what future enhancements might look like for your interface. Consider scalability and potential additions, so your users won’t be caught off guard when new features roll out.

3.Graceful deprecation with configurations
When necessary, introduce dedicated configurations to support both forward and backward compatibility. Mark the previous support as @deprecated and provide clear instructions on transitioning to the new implementation.

4. User-friendly “Turn Off” option
Offer your customers a simple way to disable a change if they struggle to integrate a new API seamlessly. Empower them to control the pace of transition.

5.Release breaking changes selectively
Only resort to breaking changes when absolutely necessary, and when you do, accompany them with a clear and comprehensive migration guide. Ensure users can smoothly transition to the new version.

6.Embrace API versioning
Maintain your APIs with versioning. Clearly define and document different API versions to manage changes effectively.

7. Keep a breaking changes wishlist
Maintain a list of potential breaking changes you’d like to introduce when the opportunity arises. This allows you to plan and communicate these changes proactively.

8.Centralize API alignment
Streamline your code by having a central location where you align the API to the latest version. Collect backward-compatible APIs and convert them to the latest format, ensuring that your business logic remains readable and adaptable.

9.Celebrate major updates
Encourage users to upgrade to major versions by offering compelling incentives and rewards.

Challenge #2. One Size Fits All vs. Tailored Tech

While some libraries cater to specific frameworks, most aim to be adaptable across a range of technologies. Achieving framework agnosticism is a significant challenge.

The fundamental question arises: Do you prioritize offering the best SDKs for a select few frameworks, guaranteeing an exceptional user experience? Or are you willing to compromise on the SDK development experience in exchange for ensuring that every customer can leverage your remarkable product?

In one of our features, the goal was to provide developers with a way to encapsulate sections of their application that should only be accessible to entitled users. Let’s examine a customized solution approach by focusing on the SDK for our Angular users.

We’ve streamlined the integration process, but it requires a deep understanding of Angular for development and maintenance, dedicated testing, ongoing support, and additional time investment in building the feature initially. This underscores the persistent challenge: each feature necessitates careful deliberation to strike the right balance and effectively deliver value despite having limited resources and expertise.

To effectively address the challenge of balancing generic SDK features with framework-specific ones, consider the following strategies:

1.Informed decision making

• Data-driven insights – Analyze framework usage data to inform your decisions.
• Effort estimation – Clearly estimate the time and effort required to adapt features to specific frameworks before making decisions.
• SDK support boundaries – Determine when to limit SDK support and be prepared to say “no.”

2.Prioritize essential features

• Identify critical features – Focus on making key features as easy to integrate as possible for a seamless user experience.
• Offer workarounds – When tailored support isn’t feasible, provide clear workarounds that may take slightly more time but are viable alternatives.

3.Document comprehensively with examples 

• Thorough documentation – Maintain detailed SDK documentation.
• Sample projects – Create exemplary projects demonstrating effective feature usage, which can compensate for the absence of a polished SDK.

4.Start with your MVP and gather feedback

• Begin with an MVP and gather user feedback
• Iterate based on feedback – Collaborate with design partners, assist with feature integration, and decide whether additional effort is warranted based on feedback.

5.Cultivate expertise

• Develop domain expertise – Look for opportunities to build up knowledge…within your team.
• Basic framework knowledge – Equip SDK developers with a foundational understanding of each framework to address common issues.
• Focus on high-impact frameworks – Invest more time in mastering the frameworks most widely used or posing the greatest SDK challenges.

6.Architectural Considerations

• Slim adapters – Build framework adapters that are minimal, containing only technology-specific components.
• Automated adaptation – Invest in infrastructure that can automatically adapt basic APIs to different frameworks, reducing the need for additional effort. We will get to it in more detail later on.

These strategies will help you strike the right balance and enhance the effectiveness of your SDK development efforts.

Challenge #3. Customization vs. Out-of-the-Box Excellence

Customers typically expect features to seamlessly integrate with their specific requirements – their application’s styling, brand identity, and user experience. Some customers may prefer extensive customization options to tailor the feature precisely to their unique needs.

At the same time, there are users who value simplicity and efficiency. They trust in best practices and appreciate easy, out-of-the-box solutions. We struggle with this tradeoff on a daily basis. Balancing customization with user-friendly defaults is always a critical consideration in our features.

Here is our sign-up form, a built-in component that we provide.

We strike a balance by catering to the most common requests. We provide a comprehensive form built adhering to industry best practices. Additionally, we offer a set of predefined configurations for controlling the layout and form structure. Full styling customization is also available for every component. For those with unique needs, we provide detailed documentation on how to implement a fully customized form using our APIs.

This approach allows us to keep our main component unified, without maintaining a fully dynamic and generic form, while still letting all of our customers have a proper solution for their needs, and even be ready to add additional configurations pretty easily on demand.

Practical Tips

1. Always share the building blocks – Share the core elements of your feature, assuming that your APIs are ready for use. This ensures you have a workaround ready when needed.
2. Dive into user experience – Invest in extensive research to find the best user experience solutions for various use cases, it might reduce additional needs like magic, and also gain your customer’s trust by following your practices and keeping up to date.
3. Expose common customization – Offer the most common customization options you can think of to make users’ lives easier.
4. Document thoroughly – Ensure your documentation covers all common use cases comprehensively.
5. Prepare for future configuration – Anticipate potential requests and make your infrastructure ready for future configurations. Be prepared to easily add customization options as needed, keeping it flexible and adaptable.

Challenge #4. Simplify SDK Infrastructure for Seamless Feature Development

Adding a new feature to be exposed in an SDK should meet some basic requirements that allow it to work well in many different situations. To make life easier for your developers, prevent mistakes, save time, encourage more people to contribute, and help different teams collaborate effectively on the same SDK, you should design your infrastructure to handle these basic requirements right out of the box. This means your developers don’t have to worry about these foundational matters, and they can focus on creating and improving specific features.

Let’s use an example from real life:

1. A robust shared components library
   As we continually release new features, we’re committed to supporting our customers’ customization preferences. However, we aim to avoid providing custom support for every new feature. To achieve this, we’ve developed a shared component library with a robust mechanism. This mechanism enables every component to seamlessly align with our predefined themes and inherit styling, color palettes, breakpoints, and more when customers provide them. By incorporating these shared components into each new feature, we empower full customization without the need for additional support.
   
2. Automated API adapters
   We’ve implemented a mechanism in every framework wrapper to transform all Redux sagas from our core package into actions in the relevant framework’s store. For example, here is how we do it for Vue.js customers:

This automates the process of exposing new APIs to clients. Developers can create new APIs without additional steps, ensuring a smoother development experience.

Practical Tips

1. Regression testing – We maintain a robust suite of regression tests for our features. This safeguards against accidental breakages, as not every developer is intimately familiar with our system’s logic.
   
2. GitHub PR template – Our GitHub PR template includes a checklist that guides developers through the necessary steps for successful integration. This template is a helpful tool, keeping developers on track and reducing the likelihood of overlooking essential tasks.
   
3. Centralized backward compatibility management – We established a centralized approach to managing backward compatibility, simplifying the process of ensuring that previous implementations remain supported alongside newer versions.

Challenge #5. Mastering Quality Assurance and Release Pipelines

Building an SDK, especially an open-source one, means that your code can impact millions of end users across various applications. The responsibility of releasing new code to production is immense, as it directly affects your vast user base. To tackle this challenge effectively, you must prioritize quality and reliability in your release pipelines.

We recommend the following best practices:

1. Maintain a detailed changelog
   Developers should be required to include changelog information with their commits. Therefore you should create a dedicated process for managing changelog entries, ensuring that each commit has a proper title and changelog selection. Establish dedicated pipelines that propagate changelog information until the final release, and maintain a changelog file to document changes clearly.
   
2. Multi-stage release pipelines
   Implement a multi-stage release process. Start by releasing your core business logic internally. Gradually release it to the public source code, with a separate pipeline to ensure code quality. Consider releasing an alpha version, which can be tested by customers when necessary or by QA teams. Only after thorough testing and validation should you manually release a new npm version.
   
3. Effective communication – Create a dedicated internal Slack channel to report and track every new version being released. Ensure that support teams are informed and aligned with the release process, keeping them fully aware of what’s happening.
   
4. Support for hot fixes – Establish a dedicated pipeline for handling hot fixes promptly, and to address critical issues efficiently.
   
5. Comprehensive E2E testing – Maintain a comprehensive end-to-end testing pipeline to thoroughly assess the functionality and reliability of your packages.
   
6. Manual sanity checks – Conduct manual sanity checks for your packages, both on a clean integration setup and on existing installations. This ensures that everything continues to work as expected. 

We’ve now addressed five of the major challenges you’ll encounter in building open-source SDKs.

If you take advantage of these recommendations and best practices, the result will be faster and highly reliable releases to meet the needs of your user base, and protect them from a security standpoint. In addition, you should find that your SDK development processes become more efficient and easier to maintain. We invite you to let us know how they work out for your team, and send us your feedback and suggestions.