Convincing Boss and Self: Using R in Production

As a data scientist or an analyst, you may have proposed or even used the R programming language for various projects. However, the question of whether R can be effectively used in production often comes up. This article aims to address concerns and highlight the practicality of using R in a production environment, using the example of Microsoft Azure Machine Learning.

Addressing Concerns

The reliability of R in a production setting is a common concern. This concern is not unprecedented, as there are many rumors and perceptions about R, often spread by sales and marketing teams of competing commercial products. However, platforms like Microsoft Azure Machine Learning handle productionalization of R scripts seamlessly, as illustrated by its capabilities to easily deploy R scripts as web services.

The process of retraining models daily can also be managed within this framework. You can separate this process into a separate service, allowing for flexibility and scalability. Azure offers a range of scalable options, including R Server, acquired from Revolution Analytics.

Why Not R?

While R is a powerful tool for research and statistical analysis, it may not be the best choice for a production pipeline, especially in the realm of machine learning (ML). Other languages, such as Python, Java, or Scala, are often preferred for production due to their robustness, scalability, and maturity in handling real-world data processing and application development. R excels in scenarios where ease of scripting and exploration is crucial but falls short when it comes to maintaining a production-ready environment.

Best Practices for Production

Implementing R in production requires adherence to best practices to ensure the integrity of the models and the efficiency of the production environment. Here are some key points to consider:

Separating Training/Testing from Production: Ensuring that the models used in production are identical to the ones used in the training phase is crucial. Any discrepancies can lead to performance issues and poor predictions. Scalability: The system should be scalable to handle the expected throughput and load. This can be achieved through careful planning and possibly using platforms like Azure that offer scalable options. Error Handling and Fault Tolerance: While R is not inherently designed for fault tolerance and error handling, integrating error management and robustness mechanisms can significantly improve its performance in a production environment. This includes implementing monitoring and logging to catch issues early.

Employing the Right Tools

When deploying R in a production setting, it is advisable to use tools and frameworks designed for scalability and robustness. One such framework is PMML (Predictive Model Markup Language), which allows models to be ported across different platforms and environments. However, for more complex models, converting to PMML or rewriting the code in another language might be necessary. This process can be time-consuming and error-prone, but it ensures robustness and reliability.

Real-World Applications and Resources

To gain more insights into how R is used in production, you can explore conferences and resources such as the EARL (Efficient Applications of the R Language) Conference. This conference focuses on the commercial application of R across various industries, providing valuable insights and real-world examples. Notably, the conference includes talks from companies like Pfizer, TIBCO, and Oracle, who have successfully integrated R into their production pipelines.

For those interested in leveraging R for production purposes, the EARL conference is a great resource. Additionally, tools like Yhat and ScienceOps can provide support in productizing and integrating R models into production applications. These tools can help you manage the complexities of production environments and ensure your R models perform consistently and reliably.

By addressing the concerns and adhering to best practices, R can indeed be successfully used in a production environment. Whether you are developing an MVP or a full-fledged production system, R offers a powerful combination of flexibility and capability that can be harnessed effectively with the right approach.