Category Archives: Python

Python Machine Learning Cookbook Second Edition

Over 100 recipes to progress from smart data analytics to deep learning using real-world datasets.

The popular Python Machine Learning Cookbook, Second Edition, will enable you to adopt a fresh approach to dealing with real-world machine learning and deep learning tasks.
With the help of over 100 recipes, you will learn to build powerful machine learning applications using modern libraries from the Python ecosystem. The book will also guide you on how to implement various machine learning algorithms for classification, clustering, and recommendation engines, using a recipe-based approach. With an emphasis on practical solutions, dedicated sections in the book will help you to apply supervised and unsupervised learning techniques to real-world problems. Toward the concluding chapters, you will get to grips with recipes that teach you advanced techniques for fields including reinforcement learning, deep neural networks, and automated machine learning.


By the end of this book, you will be equipped, through real-world examples, with the skills you need to apply machine learning techniques, and will be able to leverage the full capabilities of the Python ecosystem.

Python Machine Learning Cookbook Second Edition

 

Why Use Functions in Python?

This guest post by Fabrizio Romano, the author of Learn Python Programming – Second Edition, explores why functions are integral to developing applications in Python.

Functions are among the most important concepts and constructs of any language; here are a few reasons:

  • They reduce code duplication in a program. By having a specific task taken care of by a nice block of packaged code that you can import and call whenever you want, you don’t need to duplicate its implementation.
  • They help in splitting a complex task or procedure into smaller blocks, each of which becomes a function.
  • They hide the implementation details from their users.
  • They improve traceability.
  • They improve readability.

Let’s look at a few examples to get a better understanding of each point.

Reducing code duplication

Imagine that you are writing a piece of scientific software, and you need to calculate primes up to a limit. You have an algorithm to calculate them, so you copy-paste it to wherever you need. One day, though, your friend, B. Riemann, gives you a better algorithm to calculate primes, which will save you a lot of time. At this point, you need to go through your entire code base and replace the old code with the new one.

This is, in fact, a bad way of going about it. It’s error-prone as the process entails the risks of faulty code replacement and deletion of crucial code parts, making your algorithm inconsistent and unstable. What if, instead of replacing code with a better version of it, you need to fix a bug, and you miss one of the places? That would be even worse.

So, what should you do? Simple! You write a function, get_prime_numbers(upto), and use it anywhere you need a list of primes. When Riemann comes to you and gives you the new code, all you have to do is replace the body of that function with the new implementation, and you’re done! The rest of the software will automatically adapt, since it’s just calling the function.

Your code will be shorter as it will not suffer from inconsistencies or undetected bugs due to copy-and-paste failures or oversights.

Splitting a complex task

Functions are also very useful for splitting long or complex tasks into smaller ones. The end result is that the code benefits from it in several ways, for example, readability, testability, and reuse.

To give you a simple example, imagine that you’re preparing a report. Your code needs fetch, parse, filter, and polish data, and then perform an entire series of algorithms against the data to generate a report. It isn’t uncommon to read procedures that are just one big do_report(data_source) function. There are tens or hundreds of lines of code that end with return report.

These situations are slightly more common in scientific code; they tend to be brilliant from an algorithmic point of view but lack the touch of experienced programmers when it comes to the style. Now, picture a few hundred lines of code. It’s very hard to follow through to find the places where things are changing context (such as finishing one task and starting the next one). Do you have the picture in your mind? Good. Don’t do it! Instead, look at this code:

def do_report(data_source):

    # fetch and prepare data

    data = fetch_data(data_source)

    parsed_data = parse_data(data)

    filtered_data = filter_data(parsed_data)

    polished_data = polish_data(filtered_data)


    # run algorithms on data

    final_data = analyse(polished_data)


    # create and return report

    report = Report(final_data)

    return report

The previous example is fictitious, of course, but can you see how easy it would be to go through the code? If the end result looks wrong, debugging each of the single data outputs in the do_report function will be extremely easy. Moreover, it’s even easier to exclude parts of the process temporarily from the whole procedure (you just need to comment out the parts you need to suspend).

Hiding implementation details

Refer to the preceding example to appreciate this merit as well. As you can see, by going through the code of the do_report function, you can get a pretty good understanding without reading one single line of implementation.

This is because functions hide the implementation details, meaning you don’t need to delve into the details if you don’t want to, as against the case where do_report were just one big, fat function. This reduces the time you spend reading the code. Since, reading code takes longer than actually writing it in a professional environment, it’s very important to reduce it by as much as we can.

Improving readability

Coders sometimes don’t see the point in writing a function with a body of one or two lines of code, so let’s look at an example that shows you why you should do it.

Imagine that you need to multiply two matrices:

Take a look at the following two code snippets and decide for yourselves which one is easier to read:

# matrix.multiplication.nofunc.py

a = [[1, 2], [3, 4]]

b = [[5, 1], [2, 1]]

c = [[sum(i * j for i, j in zip(r, c)) for c in zip(*b)]

     for r in a]


# matrix.multiplication.func.py

# this function could also be defined in another module

def matrix_mul(a, b):

    return [[sum(i * j for i, j in zip(r, c)) for c in zip(*b)]

            for r in a]

a = [[1, 2], [3, 4]]

b = [[5, 1], [2, 1]]

c = matrix_mul(a, b)

In the second example, it’s much easier to understand that c is the result of the multiplication of a and b. Reading through the code is also easy and, if you don’t need to modify the multiplication logic, you don’t even need to go into the implementation details. On the other hand, in the first snippet, you would have to spend a lot of time trying to understand what that complicated list comprehension is doing.

Improving traceability

Imagine that you have written an e-commerce website. You have displayed the product prices all over the pages. Imagine that the prices in your database are stored with no VAT, but you want to display them on the website with VAT at 20%. Here are a few ways of calculating the VAT-inclusive price from the VAT-exclusive price:

# vat.py

price = 100  # GBP, no VAT

final_price1 = price * 1.2

final_price2 = price + price / 5.0

final_price3 = price * (100 + 20) / 100.0

final_price4 = price + price * 0.2

All these four ways of calculating a VAT-inclusive price are perfectly acceptable. Now, imagine that you have started selling your products in different countries and some of them have different VAT rates. This means you’ll have to refactor your code throughout the website in order to make the VAT calculation dynamic.

How would you trace all the places in which you are performing a VAT calculation? Coding is generally a collaborative task and you cannot be sure that the VAT has been calculated using only one of those forms, can you?

Here’s what you do. Write a function that takes the input values, vat and price (VAT-exclusive), and returns a VAT-inclusive price:

# vat.function.py

def calculate_price_with_vat(price, vat):

    return price * (100 + vat) / 100

Now you can import this function and use it throughout your website to calculate a VAT-inclusive price, and when you need to trace those calls, you can search for calculate_price_with_vat.

Now that you’ve understood why functions are so important, explore Learn Python Programming – Second Edition to understand the nuances of Python programming to develop efficient, stable and high-quality applications. The book is replete with real-world examples that will make the fundamentals of Python programming a piece of cake and is hence a must-have for all beginners.

Python online help

To consult the Python online help we need to type the help () command to receive information on the use of the python interpreter. After issuing the command, you receive a welcome message from the online help function that invites us in case we were beginners to see the tutorial available on the Internet at the following url:

http://docs.python.org/tutorial/:

>>> help()

Welcome to Python 3.6's help utility!

If this is your first time using Python, you should definitely check out
the tutorial on the Internet at http://docs.python.org/3.6/tutorial/.

Enter the name of any module, keyword, or topic to get help on writing
Python programs and using Python modules. To quit this help utility and
return to the interpreter, just type "quit".

To get a list of available modules, keywords, symbols, or topics, type
"modules", "keywords", "symbols", or "topics". Each module also comes
with a one-line summary of what it does; to list the modules whose name
or summary contain a given string such as "spam", type "modules spam".

In the online help, as already mentioned with the help () command, just enter the name of any module, keyword or topic to get help in drafting Python programs. To exit the online guide to return to the interpreter, simply type “quit”. Instead, to obtain a list of available modules, expected keywords or usable topics, you will need to type “modules”, “keywords”, or “topics”.

Each module then has a summary contained in the online help in which all its features are listed, while to list the modules whose synthesis contains a given word we will have to add it to the word modules. For example, to get information on the array module we will insert this name in the shell of the online help to get the information shown in the figure.

Python online help

It is therefore advisable to consult the online help of the Python interpreter whenever you find yourself having to use a Python resource that you do not know adequately; we have seen that through the help of the Python Interactive Shell it will be easy and immediate to obtain sufficient documentation.

USEFUL LINKS FOR THE TOPIC:

 

Python Interactive Shell

To start Python or more correctly to open the Python Interactive Shell just click on the Start menu to find the shell icon ready in the frequently used programs, or just click on All Programs then on the entry Active State Active Python 3.1.

After activating the Python Interactive Shell, we will find a window where you can type our python instructions from the command line. Let’s start with the classic message that programmers use to send to the shell to test its regular operation; I refer to the most classic of messages: “Hello World”.

To display a message from the shell it will be necessary to print it, and then what could be the command that allows us to do this if not print (which translates into Italian means printing), this confirming the fact that reading the Python code is equivalent to reading a common listing in English.

Then to display the message “Hello World” at the command prompt, just type the following statement:

print ('Hello World')

to obtain the printout of the message as shown in the figure.

python prompt

Having done this, let’s see now how to receive a first and immediate help from the Python interactive shell; in fact, when the shell opens, the following message is displayed:

Python 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:57:36) [MSC v.1900 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.

>>>

USEFUL LINKS FOR THE TOPIC:

How to define strings in Python

A string is identified through the use of quotation marks. In the definition of a string in Python both single and double quotes can be used: “string”.

This double possibility of using the identifier allows us to include a type of quotation marks in a string enclosed by quotation marks of the other type, which is indispensable for example in the use of the apostrophe.

To better understand what has been said, let’s see an example:

>>> print “String containing the apostrophe …”

How to define strings in Python

Let’s see in detail how many ways it is possible to define a string:

  •    With single quotes: ‘String in Python!’
  •    With double quotes: “String in Python!”
  •    Escape sequences: “Type the command \” ls \ “”
  •    Through the use of transformation functions from other types of values: str (1111), str (11.11), str (11.1 + 1j)
  •    Finally the multiline strings with the adoption of three quotes (single or double).

To deep this argument:

Arithmetic operators in Python

In this post, let’s see how to perform simple calculations in the Python environment. Suppose we want to use the Python prompt as a simple calculator, then we will write:

>>> 6 + 5
11

Here we have visualized the result of an arithmetic operation: the sum of five and six. Python recognizes the numbers and the addition sign and adds them. Then show the result.

All arithmetic operators can be used:

  • addition (+)
  • subtraction (-)
  • multiplication (*)
  • division (/)

We can then combine different operations to get multiple expressions:

>>> ((5 * 4) + (6 – 3)) / (1 + 4)
4.6

In the expression we have just seen, it is possible to notice how Python uses parentheses to perform operations on numbers, these determine variations in the order in which they are performed.

Arithmetic operators in Python

Let’s see what happens if you write the same sequence without the brackets:

>>> 5 * 4 + 6 – 3/1 + 4
27.0

As you can see the result is quite different and depends on the fact that Python calculates the multiplications and divisions before the sums and subtractions, according to what is dictated by the rules imposed by the algebra.

These rules are imposed by all programming languages ​​and are used to determine the evaluation sequence of operations that goes by the name of precedence between operators.

Comments in Python

Commenting on Python is an operation that in some ways is very different from other languages, but it is quite easy to get used to this new way of inserting the explanatory text in our codes.

In Python there are basically two ways to comment on a program:

  •   single line of comment
  •   multiple line of comment

The single comment line is used to insert a short comment (or for debugging), while the multiple comment line is often used to describe something much more detailed.

Let’s see then some explanatory examples to better understand the concepts introduced up to now. Let’s start with the single line of comment:

print ("This is not a comment")
#print ("This is a comment")

Then when the interpreter encounters the symbol # (hash) ignores everything following the symbol until the end of the line. We could also write like this:

print ("This is not a comment") # Printing a text string

As for the multiple comment line instead we will use the symbol “‘, let’s see how:

" '
print ("This is not a comment")
print ("Additional comment line")
" '
print ("This is not a comment")

The comments are a useful resource for the programmer because they allow us to insert very valuable explanatory text in the optics reusability, but they are also particularly effective in the code debugging phase where the insertion of text strings helps in identifying possible bugs in our program.

Python prompt

In order to be immediately operative, we start to see the simplest of the programs we can write: that is, a simple sequence of commands.

The Python prompt will be of the type:

Python 3.3.2 (v3.3.2: d047928ae3f6, May 16 2013, 00:06:53) [MSC v.1600 64 bit (AM
D64)] on win32
Type “help”, “copyright”, “credits” or “license” for more information.
>>>

So let’s start working with the Python prompt, writing the individual commands directly, then print the programmer’s greeting on the screen:

>>> print (‘Hello world!’)
Hello World
>>>

How you can verify the print command does nothing but ask Python to show us on the video the string contained in the brackets. In our case it will write the sequence of characters ‘Hello world’.

python prompt

This sequence of characters is defined in the jargon of the character string programmers or simply string.

Python software

Let’s first see where to find the Python software to install on our machine in order to start programming with Python. The packages that we will have to install are available on the official website of the language at the following url:

http://www.python.org/

Naturally, before downloading the software versions, we will have to inform ourselves about the type of machine at our disposal and the operating system installed on it. However, remember that Python is available for practically all operating systems in circulation.

The Python interpreter will allow the translation of our Python code into a language that our computer can understand and that will allow it to execute the instructions contained in it. In the writing period of this book, the current version of the Python interpretation is 3.1.2 which represents the stable one and that is why in the examples that will accompany us in the following chapters we will refer to that version.

We can safely download the Python software from the network, as the programming with Python is all open source and therefore freely downloadable from the internet where we can find sites that in addition to providing detailed procedures for downloading the current version of the software also provide satisfactory documentation.

 

Python portability

One feature that determined Python’s success in the global programming landscape was its extreme portability; there are versions of Python for Linux platform (its natural environment since it was developed here), for Unix-like operating systems, Mac OS, and Windows in all its versions.

The versatility of Python manifests itself in its entirety when there is a Python version also for the obsolete operating system with an MS-DOS-oriented interface.

Python portability

But the development of Python does not stop at the platforms just seen but extends to those that are widely used today for the programming of mobile phones, satellite navigators and so on. Last but not least, Python can be used on Java and in .NET virtual machines.