Databricks framework for managing machine learning projects will go to an open governance model Credit: kohb / Getty Images Databricks, the company behind the commercial development of Apache Spark, is placing its machine learning lifecycle project MLflow under the stewardship of the Linux Foundation. MLflow provides a programmatic way to deal with all the pieces of a machine learning project through all its phases — construction, training, fine-tuning, deployment, management, and revision. It tracks and manages the the datasets, model instances, model parameters, and algorithms used in machine learning projects, so they can be versioned, stored in a central repository, and repackaged easily for reuse by other data scientists. MLflow’s source is already available under the Apache 2.0 license, so this isn’t about open sourcing a previously proprietary project. Instead, it’s about giving the project “a vendor neutral home with an open governance model,” according to Databricks’s press release. Projects for managing entire machine learning pipelines have taken shape over the past couple of years, providing single overarching tools for governing what is typically a sprawling and complex process involving multiple moving parts. Among them is a Google project, Tensorflow Extended, but better known is its descendent project Kubeflow, which uses Kubernetes to manage machine learning pipelines. MLflow differs from Kubeflow in several key ways. For one, it doesn’t require Kubernetes as a component; it runs on local machines by way of simple Python scripts, or in Databricks’s hosted environment. And while Kubeflow focuses on TensorFlow and PyTorch as its learning systems, MLflow is agnostic — it can work with models from those frameworks and many others. Related content analysis Beyond the usual suspects: 5 fresh data science tools to try today The mid-month report includes quick tips for easier Python installation, a new VS Code-like IDE just for Python and R users, and five newer data science tools you won't want to miss. By Serdar Yegulalp Jul 12, 2024 2 mins Python Programming Languages Software Development analysis Generative AI won’t fix cloud migration You’ve probably heard how generative AI will solve all cloud migration problems. It’s not that simple. Generative AI could actually make it harder and more costly. By David Linthicum Jul 12, 2024 5 mins Generative AI Artificial Intelligence Cloud Computing news HR professionals trust AI recommendations HireVue survey finds 73% of HR professionals trust AI to make candidate recommendations, while 75% of workers are opposed to AI making hiring decisions. By Paul Krill Jul 11, 2024 3 mins Technology Industry Careers how-to Safety off: Programming in Rust with `unsafe` What does it mean to write unsafe code in Rust, and what can you do (and not do) with the 'unsafe' keyword? The facts may surprise you. By Serdar Yegulalp Jul 11, 2024 8 mins Rust Programming Languages Software Development Resources Videos