"One of the biggest challenges [we had] was managing the pipelines and the process itself because we had 40 to 50 different pipelines. Depending on the exact use case or what kind of data we’d like to output, we could have different combinations for running them to get different outputs. So basically, the entire system isn’t so simple."
"When working with projects that had thousands of runs, loading the interface and sorting through data was super slow in Weights & Biases. Neptune is better at handling large-scale. We’re happy with this choice."
"Weights and Biases went from being reasonably priced to being way too much. Especially since more than half the people we wanted to be able to see our models weren’t doing modeling. When we looked for an alternative, Neptune was the only one that could offer us everything we needed."
"For our company, a big plus of Neptune is the availability of a self-hosted version. We were looking for such a solution and found that not many services offer this option at an affordable price. Using the self-hosted version of Neptune is no different from the cloud version for end users. However, it has a several of advantages – unlimited storage and isolation from the global network, which increases data security."
“I used to keep track of my models with folders on my machine and use naming conventions to save the parameters and model architecture. Whenever I wanted to track something new about the model, I would have to update the naming structure. It was painful. There was a lot of manual work involved. Now everything happens automatically. I can compare models in the online interface that looks great. It saves me a lot of time, and I can focus on my research instead of keeping track of everything manually.“
"MLflow requires what I like to call software kung fu, because you need to host it yourself. So you have to manage the entire infrastructure — sometimes it’s good, oftentimes it’s not."
"I’ve used Neptune from 2019, first for my personal projects and now within the company. During this time, I saw changes and improvements in UI, but also performance and reliability. But at the same time, I always appreciated that it never became too cluttered with too many things. It’s straight to the point and it’s very effective in what it does."
"At some point, one of my students tried doing the tracking process manually, and he was very frustrated after one project. Any manual change can mess up information organization and how you track it. And if you do not build it well, then you suffer, you need to recode, etc. I think it’s just a waste of time."
"Clearly, handling the training of more than 7000 separate machine learning models without any specialized tool is practically impossible. We definitely needed a framework able to group and manage the experiments."
"We primarily use Neptune for training monitoring, particularly for loss tracking, which is crucial to decide whether to stop training if it’s not converging properly. It’s also invaluable for comparing experiments and presenting key insights through an intuitive dashboard to our managers and product owners."
"We use Neptune for keeping track of all our research work and monitoring of on-going model training. Since everything is tracked in Neptune it is super easy to keep track of what we did, how we did it, and what the results were. It makes it a lot easier also direct future research directions."
"As our company has grown from a startup to a sizeable organization of 200 people, robust security and effective user management have become increasingly evident and vital."
"Neptune’s UI is highly configurable, which is way better than MLflow."
“Without information I have in the Monitoring section I wouldn’t know that my experiments are running 10 times slower as they could. All of my experiments are being trained on separate machines which I can access only via ssh. If I would need to download and check all of this separately I would be rather discouraged. When I want to share my results I’m simply sending a link.“
"The killer feature in Neptune is custom dashboards. Without this, I wouldn’t be able to communicate my simulations to Developers, the Analytics team, and business stakeholders without any hassle. Neptune gives our Data Scientists the piece of mind that their best results won’t be lost and that communication will be a breeze."