Here at Wayfair, we are faced with challenges that the average iOS developer likely never has to come across. In the last eight months, we’ve had 47 different developers make 2,000 commits into our repository. Over the past six years, this monumental amount of changes has led to a codebase consisting of thousands of files and nearly 430,000 lines of code. Each day that goes by, we have around 10-15 merge requests go into our master branch, and 50 more open at any point in time. All of this code takes a toll on the compiler, and even though it has been extremely gradual over years of work, it has become increasingly easy to see that app compile times are slowing down our productivity exponentially.
Mitigating the effects of this problem has become one of our most important goals this quarter, and will continue to be for the near future. After a lot of time and debate, we came up with a few possible strategies to reduce compile times, ranging from enforcing best practices to fine-grained compile time optimizations. We had implemented systems around each of these strategies, but we still had no concrete answers to our problem, and compile times were only slightly improved. We had to look elsewhere for a big win, and we’ve finally landed on an option we believe will work: Modularization.
Modularization: What It Is and Why All Apps Should Consider It
So you’re probably wondering; What is modularization exactly? In its simplest form, modularization is the process of breaking a codebase down into small, focused, and shareable pieces, or modules. We have put a lot of time and planning into modularization, but we did not come up with this concept on our own. Apple has been going down this path of small, focused, shareable pieces via app extensions like messaging, today, and notifications for years. All of these shareable pieces are available for any developer to use with their apps, just as ours would be if we were to open source our module, for example. Once we began looking at our large scale app architecture, we figured out very quickly that it would be beneficial to break down some pieces of the app which were used throughout most of our features.
It was incredibly useful to be able to create modules like our networking layer, model layer, and shareable UI Components, and we’ve been reaping the benefits of reusability, reduced compile times, and succinct test suites for months. A simplified example of our architecture is shown below, where any single smaller piece could be easily imported into any new app or framework we create. At the time of this original modularization effort, we were not originally focused on the compilation time aspect, and were more intent on the reuse of these modules wherever we needed them. After our “proof of concept” with some of these core frameworks, we were ready to move on to our first feature modularization!
Wayfair’s Room Planner: A Case Study
Wayfair’s Room Planner feature is a way to plan out any room in a user’s home, only using your smartphone. A user can take a picture, and is then able to use furniture models from Wayfair to “see” how the furniture would look in their room before ordering an item. Lately, there has been a push for more of our our apps coming through the pipeline to take advantage of this feature. We knew we didn’t want to completely copy all of the files of a large feature into each new place we would be using it, so this was used as our first attempt at the modularization of an entire feature. Our most immediate need at this point had come by way of easily reusing our Room Planner code in multiple apps, but there were a plethora of other reasons to do so:
- Improving local compile and remote continuous integration (CI) times throughout the team
- Splitting up tests to run small schemes purely related to the code changed, rather than running 1,500 unrelated tests together
- Establishing a greater sense of code ownership within our larger team
- Reducing the nightmares of merge conflicts in a single, large Xcode project file
- Creation of new demo/showcase apps for improving the speed of development on a single feature
Before modularizing our Room Planner feature, a best case scenario would mean it taking approximately five minutes to build our app from a clean state. Incremental builds took about two minutes, and with less powerful MacBooks, up to 4.5 minutes on nearly every change we made within the Wayfair app.
If we take our developers currently working on Room Planner as an example, they were averaging around 30 compilations per day, or roughly speaking, about an hour in compile time. On top of this, once a feature or bug fix was submitted, CI times were north of 45 minutes. Anyone can see that this is an incredible waste of developer resources and our project managers physically cringed when faced with these statistics.
Modularization of a Feature: The Benefits
For the small cost of a two week period of a developer’s time, we were able to completely modularize the first feature of our codebase and create a demo app for development. This process will become faster as we learn the most efficient way to go about modularization overall. We moved the code into its own separate Xcode project containing a static library target, a test target, a demo target, and a resources bundle. Now that the code is broken up into smaller pieces, it is much easier to track code quality and get code coverage statistics for individual sections of the app.
Injecting all the dependencies we need into each feature has led to a much cleaner and more explicit API for use in any app, rather than each being so tightly coupled to the Wayfair app alone. Applying best practices to our code using
public as necessary has allowed our features to remain as loosely coupled as possible. We were also able to pull out the small subset of tests we have for each feature and run those alone when we change its code, rather than running the thousands of unrelated tests we have for the entire app every time.
As part of the process of modularization, we’ve created demo apps specifically for the working and display of the modules. Using Room Planner as our example, we have a module demo app which sees clean builds of 1 minute 46 seconds, and incremental builds of 16 seconds. Compared to the original numbers, this is an 80% improvement for any features developed for Room Planner. The same 30 compilations from before now take 8 minutes (an improvement over the 60 minutes previously), as long as a developer’s work stays within the scope of the feature. For now, this sadly will not help any time a developer needs to touch code outside of Room Planner, and it is only appropriate to use the demo app when the developer is working within the Room Planner domain. Once our modularization effort is complete, the entire Wayfair app will only build changed modules thanks to Xcode’s caching abilities. So far, the average compile time has slightly decreased on the Wayfair app, but it is not yet enough to make a major difference in a developer’s daily life. It is only about 7% faster than previous compile times, but this will improve as the app is modularized further.
Extrapolating our Room Planner demo app data out weekly, with 30 incremental builds per day as our compilation average, saves five hours of work per week, per developer. We currently have four developers on the team and are saving 2.5 days per week of their time, which can now be allocated elsewhere. We can say that a majority of our development will now be capable within modules like this when the push is complete, on top of saving thousands of hours a year in productivity.
So, how did we modularize our features in the app? While further detail will follow in a future article (soon to come!), we can run through our overall thought process as it currently stands.
We originally took using dynamic libraries for features into consideration, but after weighing out the pros and cons, we realized that adding one for every module would cause a significant increase to app startup time. We have worked hard to decrease app startup time as another part of our yearly goals, so using a static library by default was the way to go. Dynamic libraries would thus be considered only when absolutely necessary. Dynamic libraries can also be used to bundle the static libraries together into one import. In the example below, directly from the Wayfair app, we have bundled Networking, Tracking, and Authentication frameworks, along with the few third party libraries we use into one core framework. We can then import this single framework wherever we need it rather than requiring multiple dynamic libraries.
Using these static libraries does bloat our binary size a bit, so it is important to decide on a case-by-case basis which type of module you should use. In our case, static libraries make more sense for all product modules because we will have many of these, and so many dynamic libraries could balloon our app startup time back to the six seconds it once was. An app as widely adopted as ours cannot afford to have this happen, as every second saved directly correlates to a higher customer satisfaction rating and an infinitely better customer experience. There are some cases where infrastructure modules can justify the use of a dynamic library (for example, core frameworks shared between many apps), so we have made use of them as well. If you are working on an app with many modules, static libraries are definitely recommended for a majority of them. It is hard to come up with a use case where loading in so many dynamic libraries will not ultimately be a detriment to app quality.
Next, we assessed each feature’s code and mapped out all of the dependencies these had on the Wayfair app. These included all extensions, singletons, app state, etc. We abstracted all of these dependencies into protocols, loosening the coupling between our features and the overall app. This created a clean and robust “Feature API” that the rest of our codebase was able to interact with. This API keeps any feature module unaware of the backing implementations, ensuring it doesn’t depend on anything outside the code within it’s module.
The Perks of Modularization For You and Your Team
If you have read this far and you are not on a large iOS team, you may be wondering how this could help you. Using modularization, any team of any size can benefit from:
- Reduced app compilation time
- Reduced app startup time
- Cleaner and more succinct “self-documented” public interfaces for code, entirely independent of the rest of the app
- Smaller Xcode project files
- Established code ownership
- Easier onboarding of new team members
Modularizing your app allows you the option to parallelize the builds of your app targets using Xcode, and will cache all modules that weren’t touched since the last build. You will benefit from cleaner code, be easily able to assign code ownership, and all of your modules will be able to have “self-documented” code due to the public APIs you will be creating.
For example, anyone who takes a look at
ProductBrowseTrackingManager will be pretty sure that they know what that protocol is doing. Modularization provides efficient scalability, and allows a much lower barrier to entry for any new developers joining your team. Smaller compile times directly correlate to the ability to train developers quickly, leading to faster developer training, and here at Wayfair we’ve already seen the gains of speedy onboarding for Room Planner. We are currently in the process of modularizing throughout the app, and are hoping to complete this project by the end of the year.
We hope this post has helped you learn more about how modularization can benefit you, your team, and improve the quality of your app overall. If you have any questions, feel free to comment below or reach out via Twitter (@seanscal, @bsmithers11). Look out for the next installment of our modularization journey soon – a deep dive into the implementation and difficulties of modularizing with static libraries and resource bundles. Happy modularizing!