A step-by-step tutorial on how to make horrible cat art with Neural Style Transfer and Tensorflow 2.1.

With only a few hours left on December 23rd, I decided to create a last-minute Christmas gift. I selected some pictures of my cat, used a pre-trained Neural Style Transfer model to create an artistic version of them, and merged the transformed pictures into a single image.

The resulting piece of art is so vivid and colorful that it would make anyone throw up if they dared look at it for too long. And I came to this realization only after I spent 40$ on printing that thing on a canvas. Now this atrocity stares at me every time I sit in the kitchen, an endless reminder of my failed attempt at becoming an artist.

If you want to fail at cat art too, here is a step-by-step guide on how to do it with Tensorflow 2.1. The code is also available directly on github.

Neural Style Transfer

A Neural Style Transfer model takes two images as input and generates a new image. The generated image combines the content of the first image with the style of the second one.

The original approach was proposed in 2015 and was based on the idea that the representations of content and style in a Convolutional Neural Network (CNN) are separable. The model can minimize a content loss that depends on the first image and a style loss that depends on the second one. By optimizing both objectives simultaneously, a random image can iteratively be updated to combine the content and style of two distinct images.

If we have a CNN that was trained for a computer vision task, deeper layers of this model will detect more complex information (e.g. faces rather than lines). When two images with similar content are processed in a deep-enough layer of that CNN, we can expect their activations to be similar as well. Following this idea, the content loss can be defined as an L2 norm between the activations of two images in a single deep layer of the CNN.

For the style loss, the aim is to detect which texture components tend to occur together in an image. By computing a matrix of similarity over the channels of a specific CNN layer, we can obtain a "correlation" score that represents how much textures occur together or not. A style loss for a single layer can be computed by using the distance between the similarity matrix of the generated image and the one of the style image. This process is repeated for each layer of the CNN and the sum of these comparisons is used to obtain the complete style loss.

Neural Style Transfer has evolved over the years and other variants have been proposed. These newer methods can learn to generate the image directly instead, resulting in improved computational performances. To learn more about Neural Style Transfer, you can check out this Tensorflow tutorial.

Step 1: Select Your Datasets

We need two types of datasets to use the Neural Style Transfer model.

Content

The first one is the content dataset that will contain our cat pictures. For this one I went through the 50,000 pictures I have of my cat and selected the best ones.

Style

The second dataset type is the style dataset. In this case, we're looking for pictures with interesting shapes and colors that will be used to transform the style of our original cat pictures. I ended up browsing Kaggle datasets until I found these two datasets:

• Art Images: 9000 images of 5 types of art including drawings and paintings.
• Overwatch Heroes: 2,291 images of Overwatch heroes.

Step 2: Install Dependencies

The project has 3 main dependencies.

Pipfile

[packages]
pillow = "*"
tensorflow-gpu = "~=2.1"
tensorflow-hub = "*"

You can install either tensorflow-gpu or tensorflow. We will use tensorflow-hub to load the pre-trained model and pillow to process the images.

You can install these dependencies from the Pipfile.lock here.

Step 3: Load and Resize Images

We'll use the file image_processing.py to pre-process the images. The function below first loads an image from a file and transforms it into a float tensor with 3 color channels. Then, the tensor is resized to have at least one dimension (height/width) equal to 512 while preserving the aspect ratio. Finally, we add an additional axis to represent the batch size. In our case, we'll process one image at the time so the first axis will always be of size one.

image_processing.py

import tensorflow as tf


def load_image_tensor(image_path: str, max_dim: int = 512) -> tf.Tensor:
    image_tensor = tf.io.read_file(image_path)
    image_tensor = tf.image.decode_image(
        image_tensor, channels=3, dtype=tf.float32
    )
    image_tensor = tf.image.resize(
        image_tensor, (max_dim, max_dim), preserve_aspect_ratio=True
    )
    image_tensor = image_tensor[tf.newaxis, :]
    return image_tensor

For example, if we load an image of size 1393 x 1943, then image_tensor will have the shapes:

• (1393 x 1943 x 3) after decoding,
• (367 x 512 x 3) after resizing,
• (1 x 367 x 512 x 3) after adding a new axis.

Step 4: Create an Image Tensor Generator

In loader.py, we'll create a generator that calls the load_image_tensor function of the previous step. The function loops through the files of a specified image directory. Each image is converted to a tensor and is then returned to the caller.

loader.py

from glob import glob
from os.path import join
from typing import Iterator

import tensorflow as tf

from fifty_shades.image_processing import load_image_tensor


def generate_image_tensors(directory_path: str) -> Iterator[tf.Tensor]:
    file_path_regex = join(directory_path, "*")
    for file_path in sorted(glob(file_path_regex)):
        image_tensor = load_image_tensor(file_path)
        yield image_tensor

Step 5: Transform and Save Images

In model.py we'll create an adapter class for our pre-trained model to transform and save our images. The constructor of that class downloads a pre-trained model from Tensorflow Hub. You can modify the version of the model depending on which Neural Style Transfer approach you want to use. In my case I used the original approach (version 1).

model.py

from os.path import join
from typing import Union, Iterator

import tensorflow as tf
import tensorflow_hub as hub
from PIL import Image
from keras_preprocessing.image import array_to_img

from fifty_shades.image_processing import save_image_from_tensor


class NeuralStyleTransfer:
    def __init__(self):
        self.model = hub.load(
            "https://tfhub.dev/google/magenta/arbitrary-image-stylization-v1-256/1"
        )

    def predict(
        self, content_tensor: tf.Tensor, style_tensor: tf.Tensor
    ) -> Image:
        predicted_tensor = self.model(content_tensor, style_tensor)
        predicted_image = array_to_img(predicted_tensor[0][0])
        return predicted_image

    def predict_and_save(
        self,
        image_id: Union[str, int],
        content_tensor: tf.Tensor,
        style_tensor: tf.Tensor,
        save_directory_path: str,
    ) -> Image:
        predicted_image = self.predict(content_tensor, style_tensor)
        predicted_image.save(join(save_directory_path, f"{image_id}.png"))
        save_image_from_tensor(
            save_directory_path, f"{image_id}-content.png", content_tensor
        )
        save_image_from_tensor(
            save_directory_path, f"{image_id}-style.png", style_tensor
        )
        return predicted_image

    def predict_and_save_all(
        self,
        content_tensors: Iterator[tf.Tensor],
        style_tensors: Iterator[tf.Tensor],
        save_directory_path: str,
    ) -> None:
        for i, (content_tensor, style_tensor) in enumerate(
            zip(content_tensors, style_tensors)
        ):
            self.predict_and_save(
                i, content_tensor, style_tensor, save_directory_path,
            )

The predict method calls the model with the content and style images. The resulting tensor is transformed into an image with the array_to_img function of keras. The predict_and_save method saves the content, style, and transformed images after the prediction. In predict_and_save_all, we repeat this process for multiple content and style images.

I was only testing on a small image sample so I processed the images one at the time. But if you are testing on a larger set, it would be faster to make predictions for multiple images at once instead. To save our content and style images more easily, I also added the following helper function.

image_processing.py

from os.path import join

import tensorflow as tf
from keras_preprocessing.image import array_to_img


def save_image_from_tensor(
    save_directory: str, file_name: str, tensor_image: tf.Tensor
) -> None:
    save_path = join(save_directory, file_name)
    image = array_to_img(tensor_image[0])
    image.save(save_path)

Step 6: Run the Model

We now have all the pieces in place to run our model. In the file example.py, we first create a cat and a style generators that will be used to iterate over our image tensors. Then, the model is called to transform the images and save the result.

example.py

from fifty_shades.loader import generate_image_tensors


save_directory = "my/save/directory"
cat_generator = generate_image_tensors("cat/images/directory")
style_generator = generate_image_tensors("style/images/directory")
model = NeuralStyleTransfer()
model.predict_and_save_all(cat_generator, style_generator, save_directory)

Here is the result on a couple of sample images.

Step 7: Make a Canvas

If you want to print these images on a canvas like mine (hopefully yours will look better), find a photo store near you. They usually have an online tool where you can manually select the images you want to use and combine them.

Your Turn

Now try it for yourself on your own data. Go outside, steal a cat, take pictures, transform them, and build a canvas. It's your turn to ruin someone's kitchen with your art.

If you want to learn more on Neural Style Transfer, here is a list of helpful resources:

• Deeplearning.ai videos that explain the original Neural Style Transfer approach.
• A Tensorflow tutorial on Neural Style Transfer.
• The original paper: A Neural Algorithm of Artistic Style.

The Artificial Intelligence Meetup in Quebec City had over 600 participants this year and highlighted research from Google Brain, Siemens, and many other industries and universities. This second edition took place at the Port of Quebec on April 8 and included two conference tracks with 20 presentations.

The focus of this year was mostly on reinforcement learning, creativity, healthcare, smart cities, and IoT. Here is a summary of my 5 favorite talks of the meetup and a concluding remark on ethics.

Reinforcement Learning and Deep Neural Networks

Marc G. Bellemare, Senior Researcher at Google Brain and Adjunct Professor at McGill University

Marc introduced the concepts of reinforcement learning by showing how to make a "Pâté Chinois" recipe. The reinforcement learning agent starts at the first state: an empty plate. Then, the agent must choose an action to transition to a new state. In this case, an action consists of choosing an ingredient (e.g. chopped steak, corn, or potato), and the new state could be a plate with chopped steak. The goal of the agent is to choose a sequence of actions that will maximize a long-term reward, like succeeding the complete recipe.

Marc has contributed to the creation of the Arcade Learning Environment¹ that serves as an interface to hundreds of Atari 2600 games. The goal of the agent in this environment is to choose the correct sequence of actions to win the game and obtain the highest score. The agent receives the current image of the game as input (state), and chooses an action that simulates pressing a button on a joystick controller. To assess the performances of these agents in new situations, a subset of games is used to train the agents and a distinct set is used for evaluation. Since these Atari games are considerably diversified and were created by independent groups, they provide an interesting way to benchmark reinforcement learning agents and assess their general competency.

In 2015, Marc and his team developed DeepQ-network (DQN)², an agent that combines reinforcement learning with Convolutional Neural Networks (CNN). Previous reinforcement learning approaches were mostly based on linear representations and required to manually define how to transform an image into a relevant set of features (i.e. feature engineering). In contrast, CNNs are known to perform especially well on image classification tasks and can derive their own feature representation of the image. The combination of a reinforcement learning agent with a CNN produced an end-to-end agent that became more robust to changes across different games.

Duckietown

Liam Paull, Assistant Professor, Computer Science and Operational Research Department, Montreal University

As a researcher working on autonomous cars, Liam has spent a lot of time on the engineering problem rather than the algorithmic one. He went on to find a solution to test his research in a simpler environment. Here was born Duckietown, a miniature model of a town where citizens are ducks. The autonomous car consisting of a Raspberry Pi with a camera must navigate the roads of Duckietown while respecting the traffic lights and avoiding pedestrians. The environment is open-source, reproducible at home, and competition benchmarks are available. Live Duckiebot competitions will be held at ICRA and NeurIPS this year.

The Construction of Generative Musical Models and How to Use them Creatively

Pablo Samuel Castro, Researcher, Google Brain

Pablo is a researcher and musician who uses machine learning models to generate music. He created the LyricAI system that helped the musician David Usher write new lyrics.

While the previous version of LyricAI was based on a Recurrent Neural Network (RNN), the new version³ currently relies on Transformer models⁴. Transformer models don't need recurrent layers (as used by RNN), but require instead a self-attention mechanism. This mechanism is especially important to help the model look at similar words far in the past of the sequence and keep the next generated words coherent (i.e. long-term dependencies).

Lyric AI uses two Transformer models. The first one is used to generate the grammatical structure of the lyrics. This model is trained on a lyric dataset where a lyric line is fed as input, and the model must predict the Part-of-Speech (PoS) tags of the next line. The second model is used to generate the words in the lyrics. In that case, the model is trained on a book dataset where sentences are divided into two parts. Given the words in first part of the sentence and the PoS tags of the second part, the model must predict the sequence of words appearing in the second part of the sentence.

Pablo ended his talk by doing a live demo of creative machine learning models. He used an RNN model to generate drum beats, started a piano improvisation, and let another model generate the remaining piano melody.

AI (ML) and Healthcare Now and the Future

Anna Goldenberg, Senior Scientist at the Hospital for Sick Children and Assistant Professor at the University of Toronto

Machine learning has many applications in healthcare. Anna and her team have used machine learning approaches to tailor treatments to similar groups of individuals⁵, predict the age of cancer onset for people at high risks⁶, and predict malignancy of thyroid cancer to reduce unnecessary surgeries by 20%.

Despite these advances, there are different problems with machine learning models that prevent them from being used in healthcare. One reason is that policies in place can interfere with the data. For example, patients with asthma start with more aggressive treatments for pneumonia because of their higher risks. A model trained on the outcome of patients could then wrongly infer that asthma leads to a higher chance of survival. Studies are also required to compare the prediction of a model with those of the experts to prove that the model causes no harm before being deployed. Once the model is deployed, updates can be harder to implement because of FDA regulations, leading to lower performance over time.

Bandit Algorithms and their Application to Adaptive Clinical Trials

Audrey Durand, Postdoctoral Student, McGill University

Audrey used an interactive reinforcement learning agent to select efficient treatments for mouses with skin cancer⁷. The treatments in this clinical trial were assigned in a sequential way rather than at the beginning of the study. Hence, the effects of a treatment on the first patient were used to help the agent choose a treatment for the following patients.

The agent assigned treatments by estimating the probability of success of different options and compromising exploration and exploitation phases. The agent exploits when it assigns the optimal treatment according to current information. In contrast, the agent explores when it assigns a treatment to acquire additional information.

During the trial, the life expectancy of mouses increased over time and the variance diminished. This behavior is expected because the time spent in the exploration phase decreases as the study progresses. The agent also learned to alternate two treatments over time, which helped the mouses to recover from high doses of chemotherapy.

A Closing Note on Ethics

The meetup ended on a subject that is becoming increasingly popular among the AI community: ethical impacts of AI systems. François Laviolette, Director of the Big Data Research Center at Laval University, presented different ethical concerns surrounding AI systems. One example is the Cambridge Analytica scandal where personal data of millions of Facebook users was used without their consent to influence U.S. votes. Other concerns include fairness, privacy, and accountability of AI systems, as well as their impacts on job security.

To address these concerns, François and Lyse Langlois announced the creation of the Observatory on the Societal Impacts of AI. The observatory regroups 18 educational institutions and 160 researchers who will work to minimize the negative impacts of AI systems. If you or your company are interested in developing responsible AI systems, you are invited to sign the Montreal Declaration for a Responsible Development of Artificial Intelligence.

References

A desire to increase opportunities for software developers to learn and share their knowledge in Quebec City ignited the creation of Cabane.io, a new conference for developers. The idea that originated from a conversation over a beer was concretized on January 19 when 120 participants entered the Musée national des beaux-arts du Québec for the first edition of Cabane.io.

The conference created by Vincent Seguin and Guillaume Simard, two software developers from the region, instantly captured the interest of the community and was sold out in less than 7 hours. The presentations were rich in live demos and code examples with subjects covering programming languages, front-end development, machine learning, DevOps, and more.

The conference was comprised of 6 talks of 45 minutes and 5 lightning talks of 10 minutes. Here is a quick overview of all the presentations with links to the videos, slides and code when available.

Some slides are only available on the slack channel of Cabane.io, you can create a free account here to access them.

Main Conference Talks

React at Scale

William Fortin, Engineering Lead, Pleo.io

William (a.k.a. Martha Stewart) showed a live recipe to convert JavaScript code into TypeScript, and took pre-baked code from his oven to present Higher-Order Components, Render Props, and Hooks with React. With his live demo, William showed that enterprises can gradually adopt React by iteratively focusing on small rewrites.

video  slides   code

Elixir, a Functional Remedy to Web Development

Guillaume Cauchon, Software Engineer, Mirego

Elixir is a functional language designed for scalability and maintainability with a syntax inspired by Ruby. By diving into the language, configurations, tests, and deployment, Guillaume showed that Elixir might just be the solution to propel your Web application to the next level.

video  slides   code

Stream Processing on AWS Using the Kappa Architecture

Joey Bolduc-Gilbert, Software Developer & Team Lead, XpertSea

XpertSea analyzes information about aquatic organisms by using IoT devices to collect aquaculture data. Joey presented the core system of their data platform based on the Kappa Architecture for real-time stream processing, a simplified approach that avoids relying on a batch processing layer and where the code base depends instead on a single processing framework.

video  slides

Neural Network in Python

Carl Chouinard, Chief AI Officer, Vooban

Carl presented a Python implementation of a Neural Network created from scratch to classify images of numbers using the well-known MNIST dataset. From layers and activation functions to back-propagation and evaluation, Carl gave an overview of the building blocks behind Neural Networks.

video

Creating a Terraform Provider

André Thériault, Software Developer, Coveo
Maxime Coulombe, Software Developer, Coveo

Terraform is an open-source tool that uses declarative configuration files to create, manage, and update infrastructure resources. André and Maxime showed the power of Terraform by creating a custom Coveo provider and making a surprising revelation about a dreadful situation where the simplicity of a Terraform rollback saved the day.

video  slides

Code Review Like a Boss

Marc-Antoine Aubé, Programmer II, Poka

Code reviews are a way to collaborate, share knowledge, and improve the code base, but they don't always result in a positive experience. From a reviewer, an author, and a team standpoints, Marc-Antoine shared the best practices to transform code reviews in a learning and mentoring experience that everyone can enjoy.

video  slides

Lightning Talks

Customization with Vue.js

Jean-Sébastien Tremblay, Developer, Snipcart

Snipcart is an HTML and JavaScript-based shopping cart that can be easily integrated in a Web application to add e-commerce functionalities. Jean-Sébastien presented how Snipcart made the switch to Vue.js and were able to make their shopping cart overridable for customization purposes.

video

Deployment with Kubeflow

Mathieu D'Amours, Chief Technology Officer, Braver

Kubeflow combines the strength of Kubernetes and Tensorflow to simplify the deployments of machine learning workflows. Mathieu presented the different components of Kubeflow by diving into the core concepts behind Kubernetes, creating Tensorflow models with Jupiter Notebooks, and showing a quick example of Kubeflow at work.

video  slides

Templating with lit-html

Philippe Desjardins, Software Developer, XpertSea

Philippe wants to create an amazing application to like and dislike different types of Cabane, but he's not impressed by current Javascript frameworks and libraries like React, Vue.js, and Angular. Armed with lit-html, a new library to write HTML templates in JavaScript, Philippe created an efficient Cabane application right before our eyes.

video  code

IoT to Server to WebSockets

Sébastien Légaré, Team Lead, Vooban

Keeogo is a lower-body exoskeleton designed to support those with mobility difficulties during daily activities. Sébastien presented how his team used WebSockets to solve the challenging task of connecting the Keeogo to the cloud for device updates, components integrity verification, and users statistics visualization.

video

Don't Get Oxidized with Rust

Pierre-Alexandre St-Jean, Senior Software Engineer, Pleo.io

Pierre-Alexandre is a hipster programmer who worships the programming language Rust. By impersonating a functional developer, an object-oriented programmer, and a Lisp guru, Pierre-Alexandre showed that Rust is adapted for different programming paradigms but there are many ways you can shoot yourself in the foot.

video  slides

Want to know more about Cabane.io? Consult their website, join the slack channel or follow them on Twitter, Facebook, and YouTube.

More than 500 attendees gathered last month for the Artificial Intelligence Meetup in Quebec City. Presentations from researchers and industry practitioners were focused on deep learning, natural language processing, computer vision, and industrial applications of artificial intelligence. This post presents a summary of each technical talk of the meetup.

Outline

1. Artificial Intelligence and Deep Learning - Yoshua Bengio
2. Artificial Intelligence: Between Potential and Possible - François Laviolette
3. Overview of the AI Ecosystem in Quebec - Christian Gagné and Alexandra Masson
4. Coveo Machine Learning: Extensible Machine Learning Platform for Personalized Predictions - Sébastien Paquet
5. How AI Helps Gartner Know (Almost) Everything on the Job Market Worldwide - Andriy Burkov
6. Health and Toxicity in Online Conversations - Richard Khoury
7. SCALE.AI : AI-Powered Supply Chains - Louis Roy
8. Artificial Intelligence and Augmented Reality - Jean-François Lalonde
9. Towards Intelligent Nanoscopy: Artificial Intelligence Applications in the Study of Brain Molecular Mechanisms - Flavie Lavoie-Cardinal
10. Real Time Analysis of Decision Patterns and Bio-Behavioral Data for Augmented Human-Machine Systems - Daniel Lafond
11. Panel : Artificial Intelligence and Industry 4.0 in Quebec - Alexandre Vallières, Jonathan Gaudreault, Yves Proteau, Sébastien Bujold, and Kevin Spahr

1. Artificial Intelligence and Deep Learning

Yoshua Bengio, Scientific Director, Montreal Institute for Learning Algorithms (MILA)

Artificial intelligence applications are everywhere. They can converse in natural language, recognize objects in images, play complex games like Go, and even detect cancer cells in images. These advances were made possible with the evolution of AI techniques over the years. While early stages of AI were mostly focused on the formalization of knowledge, we now use machine learning methods to let computers gather knowledge by themselves from observations. Now, the source of intelligence is data.

Deep learning is a set of machine learning methods based on neural networks. Concretized by Yoshua Bengio, Geoffrey Hinton, and Yann LeCun in 2006, deep learning approaches are particularly good on perception tasks like interpreting sounds or images. Their ability to absorb a lot of information and adapt to different contexts have made them methods of choice for many machine learning tasks.

Deep learning methods create an implicit representation of the knowledge. As a result, they can transform data in a form that is more semantically interesting. This phenomenon was observed on a convolutional neural network that was trained to recognize scenes (e.g. office, restaurant) in images (Zhou, 2014). While the network had only access to a training set of images with scene labels, some computation units of the network had specialized to detect people, animals, and lighting. Hence, the model created a new representation that helped to generalize and improve its performances.

Learning representations is also useful for natural language processing tasks. Around 2000, Bengio and his team trained a neural network to learn word representations (Bengio, 2003). This trained model could transform a word into a real-valued vector such that words with similar meanings were close to each other in that vector space. For instance, a 2-dimensional representation of that space could show that "was" and "were" or "come" and "go" were neighbors in that space. This work was the foundation of word embeddings which are a key component of most natural language processing tasks today.

Deep learning algorithms can learn to generate images, music, and text. Generative adversarial networks (GANs) are a type of deep neural networks that learn to approximate the distribution of the data (Goodfellow, 2014). GANs are composed of two networks, the generative and discriminative networks, which compete against each other as adversaries. The generative network generates data and tries to fool the discriminative network, while the discriminative network acts as a cop and tries to detect whether the data is real or was generated. Both networks optimize a different objective and are trained in parallel.

The field of AI has made great progress over the years. However, we are still far from human intelligence. Most industrial applications of machine learning are currently based on supervised learning, where the machine learning model is trained on a dataset that contains the right answers for the task to learn. Conversely, humans tend to learn by interacting with their environment, in an unsupervised way.

2. Artificial Intelligence: Between Potential and Possible

François Laviolette, Director, Big Data Research Center (CRDM), Laval University

Artificial intelligence is no longer restricted to large corporations. Companies of various sizes and sectors have benefited from adding AI into their core operations. But despite the potential of AI, there are some limitations.

AI systems are no better than the data they are trained on. Microsoft's AI Chatbot Tay, which learned to converse by interacting with internet users, started to send Nazi messages and was stopped 16 hours after its launch. While Tay is an extreme example, racism and sexism can be present in the data. Originally applied for domain adaptation problems, domain adversarial training of neural networks (DANN) can help to deal with fairness issues (Ajakan, 2014). DANN can learn a new representation of the data that performs well on the task to learn, but performs poorly on a discriminative task like discriminating between men and women.

Problems can also arise when the training data is not representative of the reality. Consider a machine learning model that learns to classify images of cats and dogs. Suppose that each cat image in the training data contains a bowl of food, while each dog image contains a ball. A machine learning model trained on this data could learn to use only bowl and ball features to classify cats and dogs perfectly. The system would perform well on this particular dataset, but perform poorly in a real-life scenario.

Robustness is another issue with AI Systems. Small perturbations to input examples can make machine learning models output incorrect answers with high confidence. This phenomenon was observed on a neural network trained to classify images (Goodfellow, 2014). The model correctly labeled a panda image with a 57% confidence. However, the same model labeled the panda image as a gibbon with 99% confidence after the input pixels were slightly altered.

AI systems have to potential to impact companies in many ways. But we need to understand their limitations. Above all, we need to be careful on how we choose to use the data we collect.

3. Overview of the AI Ecosystem in Quebec

Christian Gagné, Professor, Department of Electrical and Computer Engineering, Laval University
Alexandra Masson, Director - Innovation, Québec International

Quebec City is becoming a key player in the field of artificial intelligence.

Laval University has developed an AI expertise in both fundamental research and applications. Research groups such as the DAMAS, GRAAL, and REPARTI are advancing AI research in fields like robotics, computer vision, natural language processing, and bioinformatics. Other groups like the CRDM and FORAC are working closely with the industry. The CRDM helps businesses with big data challenges and regroups researchers from five faculties.

Laval University also has an international visibility in AI. François Laviolette, Christian Gagné, Jean-François Lalonde, Mario Marchand, Philippe Giguère, and Brahim Chaib-Draa are six professors at Laval University that have contributed significantly to top AI conferences.

More than 75 companies in Quebec City use AI in their operations. These companies are operating in diverse sectors including health, security, insurance, management, marketing, and entertainment. The AI expertise in industries is also diversified and includes machine learning, sensors, and automation.

The current number of AI experts in Quebec City is not enough to fulfill the industry needs. For that reason, a new Professional Master’s Program in Artificial Intelligence will start this fall at Laval University. This program will combine fundamental AI courses with internships to help train 35 to 50 students in AI each year.

4. Coveo Machine Learning: Extensible Machine Learning Platform for Personalized Predictions

Sébastien Paquet, Team Lead - Data Analysis, Coveo

Coveo is a provider of search engines for businesses. Founded in 2005, the company currently has more than 300 employees. Their products are used by thousands of companies in 18 different languages.

Machine learning is central to Coveo's products. Their machine learning models can suggest search queries, optimize search results, make recommendations based on navigation history, and detect important terms in queries. Their infrastructure is 100% automated to train machine learning models and select hyper-parameters.

Coveo uses machine learning to improve search results. Consider a user that searches for a computer mouse with a serial number that does not exist in the system. If the search engine is uniquely based on keyword matching, no results would be returned for that search query. With machine learning, the system can learn from the queries of that user and the results he clicked. As more users search for the same serial number, the machine learning model can learn to match it with the corresponding mouse product, and thus improve the search over time.

Coveo's machine learning models can make personalized predictions. When a user is typing a query, their machine learning models can suggest a set of candidate queries to complete the prefix entered by the user. Often, the machine learning model only has a few input characters to make a prediction. To improve the suggestions, they use a clustering approach to group users into different clusters prior to the query suggestion. Hence a salesforce and an anonymous users could get different query suggestions for the same prefix.

Coveo has different machine learning projects in progress, including chatbots and e-commerce recommendations.

5. How AI Helps Gartner Know (Almost) Everything on the Job Market Worldwide

Andriy Burkov, Global ML Team Leader, Gartner

Gartner is a research and advisory firm that provides insights, advice, and tools to business leaders worldwide. Wanted Technologies, a provider of talent recruitment tools, was acquired by Corporate Executive Board (CEB) in 2015, which was later acquired by Gartner in 2017. Following the work of Wanted Technologies, Gartner uses artificial intelligence to help human resources with talent recruitment.

Artificial intelligence has many applications in talent recruitment. Gartner uses artificial intelligence to predict the ideal profile for a job, find candidates that match a specific profile, and estimate job salaries. Their AI models can predict candidate specific information such as whether the candidate is ready to leave his current employment and if he will stay long in the company. Their predictions can also help human resources estimate the duration of the recruitment process.

Gartner has an automated AI pipeline to harvest, normalize, and index job data. Their crawlers download millions of job postings from different websites. They use machine learning to extract job posting attributes like company names, locations, salaries, occupations, and dates. They can also detect duplicate posts or posts that refer to multiple jobs.

Gartner uses machine learning and natural language processing for salary extraction. They use a binary classification model to predict if a number is a salary or not. When a number is predicted as a salary, they divide the phrase into tokens for further analysis. For instance, in $2k monthly, the $ symbol is the currency, 2k is the amount and means 2000, and monthly is the period. Hence, 2000 should be multiplied by 12 to become a yearly salary.

Gartner's machine learning team is planning to work on different projects in the future, such as crawlers that can automatically understand website structures to find specific information, and a writing assistant for job postings.

6. Health and Toxicity in Online Conversations

Richard Khoury, Professor, Department of Computer Science and Software Engineering, Laval University

Toxicity is present in online conversations. Some individuals use online media to manipulate public opinion, propagate racist and sexist ideologies, and encourage suicide. Groups like ISIS use social media to expand their influence and recruit people. And with the massive volume of online messages, it becomes impossible for human moderators to monitor everything.

Artificial Intelligence can help with online toxicity. The goal is not to monitor everything, but rather to identify the individuals that should be monitored. One possible approach is to model the personality of online users, and then predict which types of personality are more likely to become toxic. Human moderators can then focus their efforts on users with a potentially toxic personality.

Traditional approach to measure personalities are challenging to apply online. Personality models like the Big Five and the Dark Triad can be used to describe personalities. However, they are typically measured by answering long questionnaires. The user could simply refuse to answer the questionnaire or answer dishonestly.

Alternatively, artificial intelligence can be used to predict user personality from online messages. Richard Khoury and his team did an experiment with 899 twitter accounts that were selected randomly to predict personality traits from Twitter messages. Their results suggested for instance that personalities with high psychopathy and Machiavellism were associated with glorifying and aggressive messages.

For their next experiments, Richard Khoury and his team plan to use more users, diversify their sources, and work with psychologists to define toxicity rating metrics.

7. SCALE.AI : AI-Powered Supply Chains

Louis Roy, President and Founder, Optel Group

Supply chains are the sequences of processes involved in moving a product from a supplier to a consumer. They include the sourcing, manufacturing, distribution, and delivery of products and services. SCALE.AI (Supply Chains and Logistics Excellence.AI) is an industry-led consortium that aims to use AI technologies to create an intelligent supply chain platform.

Supply chains are crucial to our society. They contribute to the transformation of natural resources into finished products and create a massive amount of jobs along the way. But supply chains are not perfect. We currently consume in six months what the earth can produce in a year. If we do not change our current supply chains and stop wasting natural resources, what are the impacts for our planet?

Technology can improve supply chain processes in many ways. Manufacturers can use technology to optimize their stocks and minimize their losses. Companies can improve their competitiveness by automating different processes. Blockchain can trace products back to their sources and improve transaction security. With the massive amount of data created by supply chains, artificial intelligence and other technologies can thus contribute to the improvement of supply chain activities.

8. Artificial Intelligence and Augmented Reality

Jean-François Lalonde, Professor, Department of Computer Science and Software Engineering, Laval University

Augmented reality is a combination of a real world and a virtual environment. The perception of the real world environment is altered by the incorporation of illusions that are perceived as being part of the environment. For instance, augmented reality can simulate realistic situations to train surgeons, and help architects add virtual 3D models of objects in their designs. Currently, the two main challenges of augmented reality are to adapt illusions to the movement of real-world objects and to the lighting conditions of the real-world environment.

Mathieu Garon in collaboration with Creaform worked on the real-time tracking of objects for augmented reality. Consider the problem of creating an illusion to make a red toy dragon appear blue in a real-world environment. As the dragon moves, the blue illusion must move as well to avoid showing any red on the dragon. Assuming that the 3D model of the dragon is known, the goal is to train a neural network to predict the change in position and orientation of the dragon at each time step. During prediction time, the network receives as input the current and previous images, the previous position, and the previous orientation. However, by using only these inputs, small prediction errors will accumulate at each time step and the predicted position of the object will eventually diverge from the real one. For that reason, the neural network must also use the predicted change in position and orientation as input. That way, the network can learn to correct its predictions and stop the error propagation.

Virtual objects that are illuminated with the real lighting conditions can better integrate with the real world environment. When the lighting conditions in an image are known, they can be used to estimate the illumination that must be applied on the virtual object. Marc-André Gardner worked in collaboration with Adobe to estimate lighting conditions in images (Gardner, 2017). His team used a large dataset of outdoor panoramas to train a convolutional neural network for this task. By using a panorama, they were able to obtain the ground truth of the lighting conditions. They then cut the panoramas into smaller images and trained the network with these images. At prediction time, they were able to predict lighting conditions for new images and insert virtual objects that appeared realistic.

9. Towards Intelligent Nanoscopy: Artificial Intelligence Applications in the Study of Brain Molecular Mechanisms

Flavie Lavoie-Cardinal, Researcher, CERVO, Laval University

Flavie Lavoie-Cardinal studies the molecular interactions in living cells. These interactions include the communication between neurons and the evolution and changes in synapses. With a super-resolution optical microscope, living cells can be observed in real time. However, the quality of images can vary greatly depending on the microscope parameters and the structure of interest. Since non-experts have difficulty to judge the quality of super-resolution images, selecting the best parameters for this microscope is highly challenging.

Artificial intelligence can help non-experts optimize a super-resolution optical microscope to obtain high-quality images. Flavie Lavoie-Cardinal and her team used a convolutional neural network (CNN) to learn to predict the quality of super-resolution images (Robitaille, 2018). They modeled the problem as a regression problem, where the CNN is given an input image and outputs a real-valued score between zero and one that represents the quality of the image. The CNN was trained on a set of high-resolution images that were labeled by an expert, and now runs automatically during their experiments to assist non-experts in predicting the quality of images. Non-experts can thus use the feedback of the CNN to optimize the parameters of the microscope and obtain high-quality images.

10. Real Time Analysis of Decision Patterns and Bio-Behavioral Data for Augmented Human-Machine Systems

Daniel Lafond, Specialist in Cognitive Engineering and Human Factors, Thales

Thales provides services for the avionics, defense, security, aerospace, and transportation markets. Founded in 2000, Thales currently has 61,000 employees and operates in 56 countries. Thales Quebec is their fifth technology research center and focuses on human, data, and IoT.

Thales is working on the real-time detection of critical states in humans. Workers like pilots or truck drivers must be extremely vigilant during their work. Their performance can be affected by stress, mental charge, and tiredness. With biosensors and artificial intelligence, workers can be monitored during their activities and critical states can be inferred from data. However, critical state classifiers have some limitations. Thales observed a significant drop in their classifier performances when they were not tested on the same individuals as they were trained on. Hence, the classifiers must currently be calibrated on each individual to perform well.

Thales also uses interpretable machine learning models to understand and support human decisions. Since human judgment can be affected by different factors, a statistical model of a doctor can be better than the doctor himself at making certain decisions. Expertise can also be hard to communicate. Hence, learning a statistical model from the data can be easier than developing an expert system. As an example, Joanny Grenier worked on understanding doctor decisions for the detection of sepsis in patients (Grenier, 2016). Sepsis is a dangerous disease where symptoms can vary highly among patients. She modeled the decision process of individual doctors by learning a decision tree on past decisions and constraining the model to use only the information consulted by the doctor. The trained doctor model can then be compared to a collective model with a better performance to understand in which cases the decision process differs.

11. Panel : Artificial Intelligence and Industry 4.0 in Quebec

Moderator : Alexandre Vallières, Cofounder, AIworx

Jonathan Gaudreault, Director, Research Consortium in Engineering and Industrial Systems 4.0, Laval University
Following steam power, electricity, and computer automation, technologies like AI and IoT are giving rise to the fourth industrial revolution. AI can help the manufacturing sector by supporting human decisions, detecting causes of manufacturing problems, and making predictions for different scenarios. For example, a manufacturer of Nespresso machines used a neural network to predict the optimal configuration of the lacquering process given the meteorological conditions. These predictions decreased the percentage of parts that had to be reprocessed from 60% to 5%. Hence, manufacturers can greatly benefit from adding AI technologies into their operations.

Yves Proteau, Codirector, APN
APN is a machine shop that transforms metal into high precision products. APN uses AI to predict product measurements, plan their production optimally, and predict when their machines should be maintained. They also have a robot that uses AI to avoid workers on the floor. AI has many applications in their industry, and collecting the right data is key to the success of their AI technologies.

Sébastien Bujold, Analyst - Production Systems, Aluminerie Alouette
Aluminerie Alouette is an aluminum manufacturing company that was founded in 1989. The company is currently based in Sept-Îles and produces approximately 600 000 metric tones of aluminum per year. Aluminerie Alouette is using artificial intelligence to improve their electrolysis process. They trained a convolution neural network on time series data to detect anode effects and instabilities during the electrolysis process. With their trained network, they can ensure continuous monitoring and early detection of these problems to minimize their production losses.

Kevin Spahr, Physician and Data Analyst, LeddarTech
LeddarTech specializes in the development of Light Detection and Ranging (LIDAR) sensors. A LIDAR creates a point cloud (i.e. a set of points in a 3D space) that measures the distance to a target area. The point cloud is created by sending light pulses on the target and capturing the light that was reflected. LIDARs have many applications in self-driving cars. They can for instance be used to detect pedestrians, obstacles, and objects in a given area.

References