Nowadays, AI is a crucial field with various frameworks like ML.NET that can be used to build amazing applications using pre-built models from cloud providers. It’s important to learn how these services work behind the scenes, how to create custom models, and understand how your application can interact with AI frameworks beyond just cloud providers or the source of the AI services.
How can I use ML Net?
ML Net can be used with Visual Studio 2019 or later, using any version of Visual Studio, and also can be used by Visual Studio Code, but only works on a Windows OS, Its prerequisites are:
- Visual Studio 2022 or Visual Studio 2019.
- .NET Core 3.1 SDK or later.
Image 1: Visual Studio installer, Installation Details contains the ML Net Model builder
Image 2: Visual Studio Context Menu
After adding the ML Net component to your project, you can see a wizard that allows you to set up your model as you need (Image 3).
Image 3: ML NET Wizard
Application Overview
The application starts with the weather Groups, every item contains a temperature range, a button to search the Historical data, and a forecast prediction (Image 4).
Image 4: Weather forecast main page.
The source of those groups is a table named Weather with the attributes:
- Id: primary key
- Description: that is the group description, you can see it as the title of the cards in image 4
- MinRange: Minimal temperature belongs to the group.
- MaxRange: Maximum temperature to belongs to the group.
The “History” button shows a table with all the historical data paginated. The historical data contains, the date with format (yyyy-mm-dd), the temperature, and if the day was cloudy (Image 5)
Image 5: Weather forecast historical page.
The predict option allows the users to generate their own prediction using ML Net through an API endpoint, the input data is the number of days from today that the user will predict and if the day will be cloudy (Image 6)
Image 6: Prediction page
The API result is the date, the group, and the percentage of probability that the date will belong to the group, also shows a table with the percentage of probability of every group.
Model
In the real world, there are lots of variables to keep in mind if you want to implement a Weather Forecast prediction app, such as wind speed, temperature, the season, humidity, if it was cloudy, etc.(2)
The scope of this approach is to see how ML Net can solve a custom model; therefore, a simple custom model was created, based on the temperature, and the season and if the day is cloudy, the model uses the weather as group of different forecasts, then the custom training model was designed as follow (Image 7):
- Weather (Id): Every grouper has an ID, so the label to predict it is the ID.
- Date: it is the feature of the date related to the weather
- IsCloudy: it’s a Boolean feature that indicates the relationship between weather and clouds.
- Season (Id): it is a feature that indicates the relationship between weather and season (Every season has an id)
Image 7: Training data section from ML Net wizard
You can get the data from Files, SQL Server databases, for this case, the data was collected from a View on SQL Server.
Project Architecture Overview
The weather forecast has 2 sites a front-end and a back-end, the data was stored in a SQL Server Database (Image 8). With this overall approach, the system was designed to separate the responsibilities of the business logic, the data, and the user experience.
Image 8: Sites and database
Front-end
You can find the app repository on GitHub using the following URL: https://github.com/joseflorezr/trainingangularversion
The front-end repository contains an angular 18 solution, which uses angular materials to help improve the user experience, and routing for navigation. The solution contains the following components (image 9):
- Forecast-header: The top component of the page, it shows the title with its style.
- Forecast-prediction: Contains the form for weather predictions and shows the results.
- Forecast results: Contains the historical data.
- Weather: Shows the groups of weather forecasts
- Services: Connects to the API to get weather, forecasts, and predictions
- Model: interfaces that map with the API
Image 9: Front-end components
Back-end
You can find the app repository on GitHub using the following URL: https://github.com/joseflorezr/WebApiMlNetWeatherForecast.
Image 10: Back End components
The API solution contains the following projects:
- TestWebAPi: Web API with the endpoints, contains 3 controllers, Weather, forecast, and WeatherForecast. WeatherForecast is an abstract class with the logger and the use case reference injection.
- Business: Contains the classes that contain the business logic, based on the Use Case approach(4)
- Model: It is the abstraction of the domain objects like Weather, Forecast, Season, and predicted forecast
- Data: This library contains 2 parts:
- The integration at the data level, the context with Entity Framework to get to the database.
- The integration with ML Net, after being added to the solution, some support files were scaffolded with the same name but different descriptions, in this case, the file is MLForecastModel:
- mbconfig: contains the wizard that helps to change the settings.
- consumption: a partial class that allows interaction with the model.
- evaluate: a partial class that allows to calculate of the metrics
- mlnetl: this file contains the knowledge base; it is important to share the file at the API level.
- training: Adds the training methods that support the creation of the file.
Database Project(3)
The data was built abstracting the concepts of the Weather and Season as master entities with their description, otherwise Forecast it’s the historical table that contains the information for a specific date (1 row per day) the observation, that means, the temperature, the season id and then the weather id.
Visual Studio contains a database project that allows developers to create, modify, and deploy databases, and can run scripts after the deployment. To create the ML Net model, a View named WeatherForecast was used because it’s easier to connect to the ML Net Wizard. The image 11 shows the relationship between the tables.
Image 11: Database diagram
Database projects can be deployed using the SQL Schema comparer tool, there is a post-build script that creates the data to the database model. For this app, a script was executed simulating forecast data from 1900-01-01 to 2024-06-04. The script uses random data, so the results must be different every time that you populate the forecast table.
WeatherForecast view concentrates the data used by ML Net to create the model.
API Project
The API project exposes endpoints that support getting the groups (Weather Controller), getting the historical Forecast data (Forecast Controller), and predict (Forecast Controller)
Image 12: Web API Swagger
Note: The ML net file must be added as a resource of the API because the MLForecastModel class at the moment the API uses the prediction functionality, tries to look at the file on a specific path (it could be changed).
Image 13: ML Net file location
Model Project
Contains the DTOs that can be transferred to the Front-end, basically, the weather entity has the group description and the temperature ranges, the season contains the description of the starting and end months, the forecast has the temperature, date if the day was cloudy and id, PredictedForecast inherits from forecast and the score, and weather description was added (Image 14).
Image 14: Entities
Basically, ML Net creates the MLForecastModel class, it contains the methods to use the prediction model (the result is different for the chosen scenario), but in general terms, the idea is to send an Input object (defined by ML Net) and receive results as follows:
- For a single object, use the Predict method, it will return the score for the predicted label.
- If you want to get the labels, use the GetLabels method, it will return all the labels as an IEnumerable.
- If you want to evaluate all labels, PredictAllLabels is the method, it will return a sorted IEnumerable with key-value pairs (label and score)
- If you want to map an unlabeled result, use the GetSortedScoresWithLabels, it will return a sorted IEnumerable with key-value pairs (label and score)
The PredictAsync Method (Image 15), creates the input object, starting with the user input (id, days, cloudy), it gets the projected date adds the days, and then finds the season ID based on the month (GetSeasonMethod). After the input project was complete, the chosen method to use was PredictAllLabels. In this case, the label is a Weather ID, so it was needed to get the Description from the Database for every given label.
Image 15: PredictAsync Implementation
Summary
- You can use ML NET to create your own Machine Learning models and use them as part of your API solution.
- There are multiple options (scenarios) to choose from according to your needs.
- Models can be created using diverse sources, such as Database objects, or files.
References
- https://learn.microsoft.com/en-us/dotnet/machine-learning/how-does-mldotnet-work
- https://content.meteoblue.com/en/research-education/specifications/weather-variables
- https://visualstudio.microsoft.com/vs/features/ssdt/
- https://medium.com/@pooja0403keshri/clean-architecture-a-comprehensive-guide-with-c-676beed5bdbb
- https://learn.microsoft.com/en-us/aspnet/core/fundamentals/dependency-injection?view=aspnetcore-8.0
