Spelling Correction Of The Text Data In Natural Language Processing

In this blog, we will rectify some typo errors using TextBlob, Autocorrect and pyspellchecker module.

Installation

If you have already installed TextBlob, Autocorrect and pyspellchecker then you can skip installation step.

TextBlob

pip install TextBlob

Autocorrect

pip install autocorrect

pyspellchecker

pip install pyspellchecker

Spelling Correction Using TextBlob

from textblob import TextBlob

Example 1:

text = "Machine learnning is a branch of artifecial intelligence and computer sciance."

We can observe, i have written some wrong spelling of "learning, artificial and science". Let us check TextBlob library can correct spelling mistake or not.

Attempt to correct the spelling of a blob

TextBlob(text).correct()

TextBlob("Machine learning is a branch of artificial intelligence and computer science.")

For this example, it corrected the spelling error.

Example 2:

text1 = "John is gud boy and he plays fotball"

TextBlob(text1).correct()

TextBlob("John is god boy and he plays football")

In this example, it didn't correct the spelling of good, but did for football.

Example 3:

In this example, we have text data which is list. We will convert list to pandas's dataframe and the apply lambda function to correct the spelling.

import pandas as pd

Define text data

text3 = [
    'Natural languuage procesing is a branch of computer sciance and artifecial intelligance.',
    'The Python programing language provides a wide range of tools and libraries for NLP tassks.', 
    'NLTK, an open source colection of libraries for buildding NLP programs.',
    'Spacy and Textblob libraries are also quite popular.',
    'NLP tasks are very interresting.'
]

I have entered wrong data in list to correct the typo error. Wrong spelling are:

1. languuage
2. procesing
3. sciance
4. artifecial
5. intelligance
6. tassks
7. colection
8. buildding
9. interresting

Set column width 500

This step is not required, if sentences are smaller in text data.

pd.set_option('display.max_colwidth', 500)

Convert into pandas dataframe

df = pd.DataFrame({'text':text3})
print(df)

                                                                                          text
0     Natural languuage procesing is a branch of computer sciance and artifecial intelligance.
1  The Python programing language provides a wide range of tools and libraries for NLP tassks.
2                      NLTK, an open source colection of libraries for buildding NLP programs.
3                                         Spacy and Textblob libraries are also quite popular.
4                                                             NLP tasks are very interresting.

Apply lambda function on df['text] and correct the spellings

df['text'].apply(lambda x: str(TextBlob(x).correct()))

0       Natural language processing is a branch of computer science and artificial intelligence.
1    The Python programming language provides a wide range of tools and libraries for NLP tasks.
2                        NLTK, an open source collection of libraries for building NLP programs.
3                                             Pay and Textblob libraries are also quite popular.
4                                                                NLP tasks are very interesting.
Name: text, dtype: object

Corrected the spellings of:

1. languuage - language
2. procesing - processing
3. sciance - science
4. artifecial - artificial
5. intelligance - intelligence
6. tassks - tasks
7. colection - collection
8. buildding - building
9. interresting - interesting

All wrong spelling corrected.

Spelling Correction Using Autocorrect

from autocorrect import Speller 

Example 1:

spell = Speller()

text4 = "Machine learnning is a branch of artifecial intelligence and computer sciance."

spell(text4)

'Machine learning is a branch of artificial intelligence and computer science.'

Corrected spelling of "learning, artificial and science".

Example 2:

spell1 = Speller()

text5 = [
    'Natural languuage procesing is a branch of computer sciance and artifecial intelligance.',
    'The Python programing language provides a wide range of tools and libraries for NLP tassks.', 
    'NLTK, an open source colection of libraries for buildding NLP programs.',
    'Spacy and Textblob libraries are also quite popular.',
    'NLP tasks are very interresting.'
]

df1 = pd.DataFrame({'text':text5})
df1

	text
0	Natural languuage procesing is a branch of computer sciance and artifecial intelligance.
1	The Python programing language provides a wide range of tools and libraries for NLP tassks.
2	NLTK, an open source colection of libraries for buildding NLP programs.
3	Spacy and Textblob libraries are also quite popular.
4	NLP tasks are very interresting.

df1['text'].apply(lambda x: str(spell1(x)))

0      Natural language processing is a branch of computer science and artificial intelligence.
1    The Python programming language provides a wide range of tools and libraries for LP tasks.
2                          LT, an open source collection of libraries for building LP programs.
3                                          Space and Textbook libraries are also quite popular.
4                                                                LP tasks are very interesting.
Name: text, dtype: object

Spelling Correction Using pyspellchecker

from spellchecker import SpellChecker

spell2 = SpellChecker()

text6 = "Machine learnning is a branch of artifecial intelligence and computer sciance."

text6 = text6.split()
text6

['Machine',
 'learnning',
 'is',
 'a',
 'branch',
 'of',
 'artifecial',
 'intelligence',
 'and',
 'computer',
 'sciance.']

Get the misspelled text

misspelled_text6 = spell2.unknown(text6)
misspelled_text6

{'artifecial', 'learnning', 'sciance.'}

Correct the misspelled words

for word in misspelled_text6:
    print(spell2.correction(word))
    
    #Get a list of 'likely' options
    print(spell2.candidates(word))

learning
{'learning'}
science
{'sciencey', 'sciences', 'science'}
artificial
{'artifcial', 'artificial'}

We got the corrected words.