Python (3) RegEx in Python [20210606]

We had covered Strings in our previous session, which can be found here:
Python (2) String and related packages [20210523]


In this post, we are covering the Python package "re", which stands for RegEx or Regular Expressions.

First step towards working with "RegEx" is by importing the "re" package. Without importing "re", package Python would not know what you are mean by "re".

(base) C:\Users\Ashish Jain>python
Python 3.7.1 (default, Dec 10 2018, 22:54:23) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> re

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 're' is not defined

>>> import re
>>> re

<module 're' from 'E:\\programfiles\\Anaconda3\\lib\\re.py'>
>>>

RegEx Functions At a High Level

The re module offers a set of functions that allows us to search a string for a match:

findall : Returns a list containing all matches
search  : Returns a Match object if there is a match anywhere in the string
split   : Returns a list where the string has been split at each match
sub     : Replaces one or many matches with a string

A Regular Expression composed using following parts: Metacharacters, Special Sequences, and Sets.

Metacharacters: These are the characters with a special meaning

Character: Description: Example

[]  A set of characters	"[a-m]"	

\  Signals a special sequence (can also be used to escape special characters): "\d"	

.  Any character (except newline character): "he..o"	

^  Starts with	"^hello"	

$  Ends with: "world$"	

*  Zero or more occurrences: "aix*"	

+  One or more occurrences: "aix+"	

?  Zero or one occurences: "r.?" 

{}  Exactly the specified number of occurrences	"al{2}"	

|  Either or "falls|stays"	

()  Capture and group

Special Sequences: It is a \ followed by one of the characters in the list below, and has a special meaning:

Character: Description: Example

\A	Returns a match if the specified characters are at the beginning of the string	"\AThe"	

\b	Returns a match where the specified characters are at the beginning or at the end of a word
(the "r" in the beginning is making sure that the string is being treated as a "raw string")	r"\bain"
r"ain\b"	

\B	Returns a match where the specified characters are present, but NOT at the beginning (or at the end) of a word
(the "r" in the beginning is making sure that the string is being treated as a "raw string")	r"\Bain"
r"ain\B"	

\d	Returns a match where the string contains digits (numbers from 0-9)	"\d"	

\D	Returns a match where the string DOES NOT contain digits	"\D"	

\s	Returns a match where the string contains a white space character	"\s"	

\S	Returns a match where the string DOES NOT contain a white space character	"\S"	

\w	Returns a match where the string contains any word characters (characters from a to Z, digits from 0-9, and the underscore _ character)	"\w"	

\W	Returns a match where the string DOES NOT contain any word characters	"\W"	

\Z	Returns a match if the specified characters are at the end of the string

Sets: A set is a set of characters inside a pair of square brackets [] with a special meaning:

Set: Description

[arn]	Returns a match where one of the specified characters (a, r, or n) are present	

[a-n]	Returns a match for any lower case character, alphabetically between a and n	

[^arn]	Returns a match for any character EXCEPT a, r, and n	

[0123]	Returns a match where any of the specified digits (0, 1, 2, or 3) are present	

[0-9]	Returns a match for any digit between 0 and 9	

[0-5][0-9]	Returns a match for any two-digit numbers from 00 and 59	

[a-zA-Z]	Returns a match for any character alphabetically between a and z, lower case OR upper case	

[+]	In sets, +, *, ., |, (), $,{} has no special meaning, so [+] means: return a match for any + character in the string

The findall() Function

The findall() function returns a list containing all matches.
The list contains the matches in the order they are found.

import re
txt = "The rain in Spain"
x = re.findall("ai", txt)
print(x)

['ai', 'ai']

txt = "The rain in Spain detected by Mr Jain."
x = re.findall("ai", txt)
print(x)

['ai', 'ai', 'ai']

txt = "The rain in Spain detected by Mr Jain."
x = re.findall("r.?", txt)
print(x)

".?" means zero or one occurence of any character, so we get the characters 'a' and ' '.
['ra', 'r ']

txt = "The rain in Spain detected by Mr Jain."
x = re.findall("r\w?", txt)
print(x)

"\w?" means zero or one occurence of any word character, so we get the characters 'ra' and only 'r'.
['ra', 'r']

If no matches are found, an empty list is returned.

import re
txt = "The rain in Spain"
x = re.findall("Portugal", txt)
print(x)

The search() Function

The search() function searches the string for a match, and returns a Match object if there is a match.
If there is more than one match, only the first occurrence of the match will be returned.

import re
txt = "The rain in Spain"
x = re.search("\s", txt)
print("The first white-space character is located in position:", x.start()) 

The first white-space character is located in position: 3

- - -

txt = "The rain in Spain"
x = re.search("\s", txt)
print(x) 

<re.Match object; span=(3, 4), match=' '>

If no matches are found, the value None is returned.

import re
txt = "The rain in Spain"
x = re.search("Portugal", txt)
print(x)

The split() Function

The split() function returns a list where the string has been split at each match:

import re
txt = "The rain in Spain"
x = re.split("\s", txt)
print(x)

['The', 'rain', 'in', 'Spain']

You can control the number of occurrences by specifying the maxsplit parameter:
Split the string only at the first occurrence:

import re
txt = "The rain in Spain"
x = re.split("\s", txt, maxsplit = 1)
print(x)
x = re.split("\s", txt, maxsplit = 2)
print(x)

['The', 'rain in Spain']
['The', 'rain', 'in Spain']

The sub() Function

The sub() function replaces the matches with the text of your choice:

import re
txt = "The rain in Spain"
x = re.sub("\s", "9", txt)
print(x)

The9rain9in9Spain

You can control the number of replacements by specifying the count parameter:
Replace the first 2 occurrences:
import re
txt = "The rain in Spain"
x = re.sub("\s", "9", txt, count = 1)
print(x)
x = re.sub("\s", "9", txt, count = 2)
print(x)

The9rain in Spain
The9rain9in Spain

Match Object

A Match Object is an object containing information about the search and the result.
Note: If there is no match, the value None will be returned, instead of the Match Object.

Do a search that will return a Match Object:

import re
txt = "The rain in Spain"
x = re.search("ai", txt)
print(x) #this will print an object

Print all the methods or attributes on Match object:

print(x.__dir__())

['__repr__',
'__getitem__',
'group',
'start',
'end',
'span',
'groups',
'groupdict',
'expand',
'__copy__',
'__deepcopy__',
'__class_getitem__',
'string',
're',
'pos',
'endpos',
'lastindex',
'lastgroup',
'regs',
'__doc__',
'__hash__',
'__str__',
'__getattribute__',
'__setattr__',
'__delattr__',
'__lt__',
'__le__',
'__eq__',
'__ne__',
'__gt__',
'__ge__',
'__init__',
'__new__',
'__reduce_ex__',
'__reduce__',
'__subclasshook__',
'__init_subclass__',
'__format__',
'__sizeof__',
'__dir__',
'__class__']

The Match object has properties and methods used to retrieve information about the search, and the result:
.span() returns a tuple containing the start-, and end positions of the match.
.string returns the string passed into the function
.group() returns the part of the string where there was a match

txt = "The rain in Spain"
x = re.search(r"\bS\w+", txt)
print(x)
print("span:", x.span())
print("string:", x.string)
print("group:", x.group())
print("start:", x.start())

<re.Match object; span=(12, 17), match='Spain'>
span: (12, 17)
string: The rain in Spain
group: Spain
start: 12

txt = "Mr Jain enjoyed the rain in Spain."
x = re.search(r"\bS\w+", txt)
print(x)
print("span:", x.span())
print("string:", x.string)
print("group:", x.group())
print("start:", x.start())
print("end:", x.end())

<re.Match object; span=(28, 33), match='Spain'>
span: (28, 33)
string: Mr Jain enjoyed the rain in Spain.
group: Spain
start: 28
end: 33

The "finditer()" function

>>> import re
>>> s = "Mr Jain enjoyed the rain in Spain with Ms Jain."
>>> re.finditer('ai', s)
<callable_iterator object at 0x000001899B48A9B0>
>>>
>>> fi = re.finditer('ai', s)
>>> for i in fi:
...  print(i.span())
...

(4, 6)
(21, 23)
(30, 32)
(43, 45)
>>>

Tags: Technology,Python,Anaconda,Natural Language Processing,