Tag Archives: python

Python Machine Learning Cookbook Second Edition

Over 100 recipes to progress from smart data analytics to deep learning using real-world datasets.

The popular Python Machine Learning Cookbook, Second Edition, will enable you to adopt a fresh approach to dealing with real-world machine learning and deep learning tasks.
With the help of over 100 recipes, you will learn to build powerful machine learning applications using modern libraries from the Python ecosystem. The book will also guide you on how to implement various machine learning algorithms for classification, clustering, and recommendation engines, using a recipe-based approach. With an emphasis on practical solutions, dedicated sections in the book will help you to apply supervised and unsupervised learning techniques to real-world problems. Toward the concluding chapters, you will get to grips with recipes that teach you advanced techniques for fields including reinforcement learning, deep neural networks, and automated machine learning.

By the end of this book, you will be equipped, through real-world examples, with the skills you need to apply machine learning techniques, and will be able to leverage the full capabilities of the Python ecosystem.

Python Machine Learning Cookbook Second Edition


Why Use Functions in Python?

This guest post by Fabrizio Romano, the author of Learn Python Programming – Second Edition, explores why functions are integral to developing applications in Python.

Functions are among the most important concepts and constructs of any language; here are a few reasons:

  • They reduce code duplication in a program. By having a specific task taken care of by a nice block of packaged code that you can import and call whenever you want, you don’t need to duplicate its implementation.
  • They help in splitting a complex task or procedure into smaller blocks, each of which becomes a function.
  • They hide the implementation details from their users.
  • They improve traceability.
  • They improve readability.

Let’s look at a few examples to get a better understanding of each point.

Reducing code duplication

Imagine that you are writing a piece of scientific software, and you need to calculate primes up to a limit. You have an algorithm to calculate them, so you copy-paste it to wherever you need. One day, though, your friend, B. Riemann, gives you a better algorithm to calculate primes, which will save you a lot of time. At this point, you need to go through your entire code base and replace the old code with the new one.

This is, in fact, a bad way of going about it. It’s error-prone as the process entails the risks of faulty code replacement and deletion of crucial code parts, making your algorithm inconsistent and unstable. What if, instead of replacing code with a better version of it, you need to fix a bug, and you miss one of the places? That would be even worse.

So, what should you do? Simple! You write a function, get_prime_numbers(upto), and use it anywhere you need a list of primes. When Riemann comes to you and gives you the new code, all you have to do is replace the body of that function with the new implementation, and you’re done! The rest of the software will automatically adapt, since it’s just calling the function.

Your code will be shorter as it will not suffer from inconsistencies or undetected bugs due to copy-and-paste failures or oversights.

Splitting a complex task

Functions are also very useful for splitting long or complex tasks into smaller ones. The end result is that the code benefits from it in several ways, for example, readability, testability, and reuse.

To give you a simple example, imagine that you’re preparing a report. Your code needs fetch, parse, filter, and polish data, and then perform an entire series of algorithms against the data to generate a report. It isn’t uncommon to read procedures that are just one big do_report(data_source) function. There are tens or hundreds of lines of code that end with return report.

These situations are slightly more common in scientific code; they tend to be brilliant from an algorithmic point of view but lack the touch of experienced programmers when it comes to the style. Now, picture a few hundred lines of code. It’s very hard to follow through to find the places where things are changing context (such as finishing one task and starting the next one). Do you have the picture in your mind? Good. Don’t do it! Instead, look at this code:

def do_report(data_source):

    # fetch and prepare data

    data = fetch_data(data_source)

    parsed_data = parse_data(data)

    filtered_data = filter_data(parsed_data)

    polished_data = polish_data(filtered_data)

    # run algorithms on data

    final_data = analyse(polished_data)

    # create and return report

    report = Report(final_data)

    return report

The previous example is fictitious, of course, but can you see how easy it would be to go through the code? If the end result looks wrong, debugging each of the single data outputs in the do_report function will be extremely easy. Moreover, it’s even easier to exclude parts of the process temporarily from the whole procedure (you just need to comment out the parts you need to suspend).

Hiding implementation details

Refer to the preceding example to appreciate this merit as well. As you can see, by going through the code of the do_report function, you can get a pretty good understanding without reading one single line of implementation.

This is because functions hide the implementation details, meaning you don’t need to delve into the details if you don’t want to, as against the case where do_report were just one big, fat function. This reduces the time you spend reading the code. Since, reading code takes longer than actually writing it in a professional environment, it’s very important to reduce it by as much as we can.

Improving readability

Coders sometimes don’t see the point in writing a function with a body of one or two lines of code, so let’s look at an example that shows you why you should do it.

Imagine that you need to multiply two matrices:

Take a look at the following two code snippets and decide for yourselves which one is easier to read:

# matrix.multiplication.nofunc.py

a = [[1, 2], [3, 4]]

b = [[5, 1], [2, 1]]

c = [[sum(i * j for i, j in zip(r, c)) for c in zip(*b)]

     for r in a]

# matrix.multiplication.func.py

# this function could also be defined in another module

def matrix_mul(a, b):

    return [[sum(i * j for i, j in zip(r, c)) for c in zip(*b)]

            for r in a]

a = [[1, 2], [3, 4]]

b = [[5, 1], [2, 1]]

c = matrix_mul(a, b)

In the second example, it’s much easier to understand that c is the result of the multiplication of a and b. Reading through the code is also easy and, if you don’t need to modify the multiplication logic, you don’t even need to go into the implementation details. On the other hand, in the first snippet, you would have to spend a lot of time trying to understand what that complicated list comprehension is doing.

Improving traceability

Imagine that you have written an e-commerce website. You have displayed the product prices all over the pages. Imagine that the prices in your database are stored with no VAT, but you want to display them on the website with VAT at 20%. Here are a few ways of calculating the VAT-inclusive price from the VAT-exclusive price:

# vat.py

price = 100  # GBP, no VAT

final_price1 = price * 1.2

final_price2 = price + price / 5.0

final_price3 = price * (100 + 20) / 100.0

final_price4 = price + price * 0.2

All these four ways of calculating a VAT-inclusive price are perfectly acceptable. Now, imagine that you have started selling your products in different countries and some of them have different VAT rates. This means you’ll have to refactor your code throughout the website in order to make the VAT calculation dynamic.

How would you trace all the places in which you are performing a VAT calculation? Coding is generally a collaborative task and you cannot be sure that the VAT has been calculated using only one of those forms, can you?

Here’s what you do. Write a function that takes the input values, vat and price (VAT-exclusive), and returns a VAT-inclusive price:

# vat.function.py

def calculate_price_with_vat(price, vat):

    return price * (100 + vat) / 100

Now you can import this function and use it throughout your website to calculate a VAT-inclusive price, and when you need to trace those calls, you can search for calculate_price_with_vat.

Now that you’ve understood why functions are so important, explore Learn Python Programming – Second Edition to understand the nuances of Python programming to develop efficient, stable and high-quality applications. The book is replete with real-world examples that will make the fundamentals of Python programming a piece of cake and is hence a must-have for all beginners.

Python portability

One feature that determined Python’s success in the global programming landscape was its extreme portability; there are versions of Python for Linux platform (its natural environment since it was developed here), for Unix-like operating systems, Mac OS, and Windows in all its versions.

The versatility of Python manifests itself in its entirety when there is a Python version also for the obsolete operating system with an MS-DOS-oriented interface.

Python portability

But the development of Python does not stop at the platforms just seen but extends to those that are widely used today for the programming of mobile phones, satellite navigators and so on. Last but not least, Python can be used on Java and in .NET virtual machines.

Python syntax

Python’s syntax makes it perfect for building simple DSLs for use within your application, thanks to its syntax and metaprogramming features.

We have already specified that Python is totally Object Oriented, this translates into the fact that in Python everything is an object. In this aspect it manages to outclass the language to objects par excellence and that is Java since in Python there is no distinction between basic types and objects, at the same time the object-oriented programming in Python is neither invasive nor annoying, and this is due to its greater purity.

A feature particularly appreciated by Python programmers was the possibility of using so-called blocks that represent an uncommon feature in the most widespread languages and which allow to represent different problems, even very different from each other, with clear and meaningful methods.

The Python is loved, in the same way, by professional programmers who appreciate the most advanced features, and beginners as it is particularly suitable to be learned as a first language.

In fact, thanks to its simplicity, it is possible to learn the basic notions of programming, without having to worry about innumerable conventions and aspects not strictly related to the realization of numerical algorithms.

Moreover Python is effectively expressive in that it provides many built-in data structures and types (the built-in types represent the types already defined in the language); examples are:

  • dict,
  • lists,
  • regexp,
  • whole numbers of arbitrary size.

The Python syntax means that the classes have a very detailed interface, which determines a simplicity in use in common operations.

Python features

The characteristics of Python make it a programming language above all simple, its use is learned in a few days, without presenting great difficulties thanks to a streamlined and very practical syntax; in addition, by presenting a really large number of bookcases included in the distribution and integrated with the installable ones, it is possible, as already anticipated, to realize our programs in a very short time.

Python is particularly dynamic, an example is provided by the fact that the variables can be safely used without the need to declare the type, then the interpreter will be able to identify them; all this, however, without losing precision because operations that are not allowed between different types will cause an error, ie it will never happen, in a more or less hidden way, that a value of a certain type is used as if it were of a different type (strong typing).

Thanks to this, an operation of type 1+ “2” (sum of a number and a string) will cause a runtime error, unlike languages ​​like php or perl that do not allow a similar control.

In Python it is possible to manipulate classes and methods at runtime, ie they can be added, deleted or modified. In this way many of the things that in other languages represent complex structures can in Python constitute simple methods; it is also possible to add custom classes and methods, creating Domain Specific Language (DSL). A Domain Specific Language is a mini-language suitable for a specific problem, the ideal would be to have a language for every purpose, but without the problem of having to learn it from scratch each time.