アプリ関連ニュース

Deploying a machine learning model into a web application with Django

Today, I would like to share about deploying a machine learning model into a web application using Django.

Note : In this article, I will focus only how to deploy a trained machine learning model in a web application with django rather than about the steps of machine learning processes.

First, I will create a simple ML model using multinomial Naive Bayes Classifier with the spam text dataset (spam.csv). The following code is to create spam text detection model.

First, import the necessary libraries.

import pandas as pd
import numpy as np
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import MultinomialNB

And load dataset file with pandas.

data = pd.read_csv("D:\Python\datasets\spam.csv", encoding= 'latin-1')
data.head()
Then extract features and labels using CountVectorizer.
data = data[["class", "message"]]
x = np.array(data["message"])
y = np.array(data["class"])
cv = CountVectorizer()
X = cv.fit_transform(x) # Fit the Data

let’s split this dataset into training and test sets and train the model to detect spam messages

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)
clf = MultinomialNB()
clf.fit(X_train,y_train)

And save the model.

import pickle
pickle.dump(clf,open("spamdetection_model.sav", "wb"))

This will output spamdetection_model.sav file. This file will be used in the web app later.

So, Let’s start to develop a Django web app.

mkdir Spamdetection – First make a directory named /Spamdetection for the Django project.

cd Spamdetection – Change directory into the folder created

And to create a Django project, it is needed to run the following command.

D:/Spamdetection>django-admin startproject spamdetectapp

This command create a new directory named spamdetectapp.

To run the djangoapp, the following command is typed in the spamdetectapp directory created.

python manage.py runserver

Copy and open the link http://127.0.0.1:8000/ in a web browser. You should see Django Home page.

Then create views.py inside the same folder to work for getting inputs from users. In the main project folder, create a new folder named ‘templates’ to work with html files and new folders named ‘datasets’ and ‘mlmodels’. Then move spam.csv dataset file into the datasets folder and spamdetection_model.sav file into the mlmodels folder.

Now our project folder will be like that.

Now open settings.py and add ‘templates’ to register in ‘DIRS’ list in the ‘TEMPLATES’ list.

Now inside the urls.py file, add the following codes to configure the urls.

from django.contrib import admin
from django.urls import path

from spamdetectapp import views # add this new line to import views files

urlpatterns = [
    path('admin/', admin.site.urls),
    
    # add these new two lines to configure for home page and result page
    path('', views.home, name='home'), 
    path('result/', views.result, name='result'),
]

And define two functions for home and result in views.py. And create a libs.py in mlmodels folder. In libs.py, we will define a getResult() function to get results from model prediction and use it in views.py.

views.py

from django.shortcuts import render
from mlmodels.libs import getResult

# for default home page view
def home(request):    
    return render(request, 'index.html')

# for result page view
def result(request):	
    message = request.POST['message']
    result = getResult(message)
    return render(request, 'result.html', {'result':result , 'message':message})

libs.py

import pandas as pd
import numpy as np
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.model_selection import train_test_split

def getResult(message):
    import pickle
    data = pd.read_csv("D:\Spamdetection\spamdetectapp\datasets\spam.csv", encoding= 'latin-1')
    data.head()

    data = data[["class", "message"]]
    x = np.array(data["message"])
    y = np.array(data["class"])
    cv = CountVectorizer()
    X = cv.fit_transform(x) # Fit the Data

    model= pickle.load(open("D:\Spamdetection\spamdetectapp\mlmodels\spamdetection_model.sav", "rb"))

    data = cv.transform([message]).toarray()    
    prediction = model.predict(data)
    if prediction == 'ham':
        return 'OK'
    elif prediction == 'spam':
        return 'SPAM'

Now we have done for backend. So, for frontend pages, let’s create index.html and result.html in templates folder.

Index.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Spam Text Detection</title>
</head>
<body>
    <h1>Spam Text detection</h1>
    <form action="{% url 'result' %}" method="post">
        {% csrf_token %}
        <p>Message:</p>
        <input type="text" name="message">
        <br>
        <br>
        <input type="submit" value='Predict’>
    </form>
</body>

</html>

result.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Spam Text Detection</title>
</head>
<body>
    <h1>Prediction</h1>
    The Message[{{message}}] is <b>{{result}}</b></body>
</html>

Now all is done. Once the above steps are completed, to stop and restart the server press Ctrl+C and run this command:

D:/Spamdetection/spamdetectapp> python manage.py runserver

And reopen the link in the browser. You will see the index home page.

Let’s test with some messages.

When message is ‘Hello, How are you?’, the result is as follow.

When the message is ‘You got $1000 prize at our shop.’, the result is as follow.

Hope you all enjoyed about this article.

By Asahi

このエントリーをはてなブックマークに追加


waithaw at 2021年11月16日 10:00:00

Database indexing

Database Indexing allows us to cut down the number of rows/records that need to be examined when a select query with a where clause is executed.

When a select query with a where clause is executed, every single row is checked to see if the query match the condition. This effectively means that the entire table will have to be scanned (known as the full table scan).

Without Indexing

An index is a data structure that stores the values for a certain specific column of a table and helps us avoid a full table scan. If the table is not indexed, the query will likely not recognize that the row order is optimized, so the query should search the rows linearly. That is, the query must search every row to find a row that matches the criteria. As you can imagine, this can take a long time. Examining every row is not very efficient.

This will take more and more time as the size of the table grows. With the sophistication of data, what can eventually happen is that a table with 1 billion rows is joined with another table with 1 billion rows. The query should spend twice as much time searching twice as many rows.

How can we imporve

If the database tables are in order, we can make search more efficiently without looking up all the records. For example, we are finding a user whose age is 18. If the age column is sorted , we can skip the records of age starting from 18.

In reality, the database table does not sort itself every time the query condition changes to optimize query performance. This is unrealistic. In reality, the index creates the data structure in the database. The data structure type is very likely to be a B-tree. There are many advantages of B-trees, but the main advantage of our purpose is that they are sortable and makes the search more efficiently.

How it works

Database indexes will also store pointers which are simply reference information for the location of the additional information in memory. So to be summarized, it creates a structure with the key of indexed column and the value of pointers which reference the actual database record. When a conditional query runs, it looks up the sorted index structure according to the data structure (B-Tree in this case), get the pointers and then go for the actual records.

Index cost

There is always something you need to give back if you want something. Index also requires additional memory. The more indexes you add, the more memory will be used. Don’t forget to index only the columns that are frequently queried. Otherwise, indexing each column consumes a huge amount of memory.

Secondly, the write operation will be a bit slower as it needs to create a index each time you add an entry to the table. A similar situation applies to updating or deleting data.

I think this will give you the abstraction knowledge of what database indexing looks like.

By Yuuma

このエントリーをはてなブックマークに追加


yuuma at 2021年11月15日 10:00:00

How to use Flutter font awesome icon ?

Need an icon not in Material?

The material design offers many helpful patterns, including a great set of icons. But they are not the only icons out there. Font awesome’s icon are truly, well, iconic. Luckily, with font_awesome_flutter package, including them in our flutter app.

It’s too simple so let’s build with simple app including flutter font awesome icon.

Firstly, we need to add font_awesome_flutter package in our pubspec.yaml file. 

dependencies:
  font_awesome_flutter: ^9.2.0

For use font_awesome_flutter package, we will need to import this package .

import 'package:font_awesome_flutter/font_awesome_flutter.dart';

To start the two components of Material Icons– wrapping with Icon and the inner icon_data value. Icons.menu is a default material icon. We will use this icon in our app. 

Icon(Icons.menu)

To change icon size and color: 

Icon(FontAwesomeIcons.user,
    size: 50, //Icon Size
    color: Colors.white, 
)

Use with FaIcon widget and let’s build a simple app then find out fond awesome icon usage!

return Scaffold(
      appBar: AppBar(
          centerTitle: true,
          leading: Icon(Icons.menu),
          title: const Text("Font Awesome Icons")
      ),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: [

            const Text(" Font Awesome Flutter package",textAlign:TextAlign.center,style: TextStyle(fontSize: 30),),
            const SizedBox(height: 20,),
            //brand Icons
            const Text("Brand Icons",textAlign: TextAlign.start,style: TextStyle(fontWeight: FontWeight.bold,fontStyle:FontStyle.italic,fontSize: 20),),
            Row(
              mainAxisAlignment: MainAxisAlignment.center,
              children: const [
                FaIcon(FontAwesomeIcons.amazon,color: Colors.deepOrange,size: 35,),
                SizedBox(width: 15,),
                FaIcon(FontAwesomeIcons.facebook,color: Colors.blue,size: 35,),
                SizedBox(width: 15,),
                FaIcon(FontAwesomeIcons.github,color: Colors.black,size: 35,),
                SizedBox(width: 15,),
                FaIcon(FontAwesomeIcons.twitter,color: Colors.blueAccent,size: 35,),
                SizedBox(width: 15,),
                FaIcon(FontAwesomeIcons.aws,color: Colors.deepOrange,size: 35,),
                SizedBox(width: 15,),
                FaIcon(FontAwesomeIcons.pinterest,color: Colors.red,size: 35,),
                SizedBox(width: 15,),
                FaIcon(FontAwesomeIcons.wordpress,color: Colors.deepPurple,size: 35,),
              ],
            ),
          ],
        ),
      ),
    );

Hope you enjoy flutter new things!

By Ami

このエントリーをはてなブックマークに追加


asahi at 2021年11月12日 10:00:00

APIs vs. Webhooks

An API (Application Programming Interface) enables two-way communication between software applications driven by requests. A webhook is a lightweight API that powers one-way data sharing triggered by events. Together, they enable applications to share data and functionality, and turn the web into something greater than the sum of its parts.

What is an API?

An API is like a portal through which information and functionality can be shared between two software services. The word “interface” is key to understanding an API’s purpose. Just like a web browser is an interface for a human end user to receive, send, and update information on a web server, an API is an interface that provides software programs with the same functionality. 

What is a webhook?

A webhook can be thought of as a type of API that is driven by events rather than requests. Instead of one application making a request to another to receive a response, a webhook is a service that allows one program to send data to another as soon as a particular event takes place. Webhooks are sometimes referred to as “reverse APIs,” because communication is initiated by the application sending the data rather than the one receiving it. With web services becoming increasingly interconnected, webhooks are seeing more action as a lightweight solution for enabling real-time notifications and data updates without the need to develop a full-scale API.

Next week, I will continue to share the usage of webhook in Laravel.

By tsuki

このエントリーをはてなブックマークに追加


tsuki at 2021年11月11日 10:00:00

Windows11 SE 発表

MicrosoftがWindows11 SEを発表しました。
これはSurface Laptop SEに搭載される、
教育機関向けノートPC専用のOSです。

続きを読む
このエントリーをはてなブックマークに追加


tanaka at 2021年11月10日 10:58:00


アプリ関連ニュース

お問い合わせはこちら

お問い合わせ・ご相談はお電話、またはお問い合わせフォームよりお受け付けいたしております。

tel. 06-6454-8833（平日 10:00～17:00）

お問い合わせフォーム