Deep
Learning for Cognitive Computing, Theory
(Course
code: TIES4910) 5 ECTS, Autumn Semester
Instructor:
Vagan Terziyan Email:
vagan.terziyan@jyu.fi
(Find the course and register in SISU system)
Attention: 1-st
Lecture: Monday, 4 September 2023. Time: 8:30 - 10:00. Place: Ag B122.1 (Alfa).
(Check also in Moodle)
===========================================================
NOTICE:
If any
content linked to by pages of this site has the prefix:
http://www.mit.jyu.fi/ai/
...
… or the
prefix:
http://www.cs.jyu.fi/ai/ ...
… and is
not accessible, then use the new prefix:
https://ai.it.jyu.fi/
...
… to access.
Sorry for the temporary inconvenience due to an unexpected server
change.
===========================================================
THIS COURSE IS INDEPENDENT PART OF OUR ARTIFICIAL
INTELLIGENCE COURSES’ PACKAGE; IT DOES NOT REQUIRE ANY PREQUESITIES AND CAN
BE STUDIED REMOTELY
Personal quota from
the course instructor:
“Everything will
be Artificial Intelligence soon, and, instead of being worried about it and
restricting it (which is a bad idea), come to study, develop, use, and promote
it further!”
Artificial Intelligence (AI) is a major driver of
economic growth and social progress, if industry, civil society, government,
and the public work together to support development of the technology, with
thoughtful attention to its potential and to managing its risks, as it mentioned
in the Report on the future of AI
from the White House Office of Science and Technology Policy. One of the most
emerging trends within AI nowadays and in observable future is Cognitive Computing and its enabler – Deep Learning.
The pair of
courses: this one – TIES4910-Deep
Learning for Cognitive Computing, Theory and its extension – TIES4911-Deep Learning for Cognitive
Computing for Developers (5+5=10 credits, delivered in English) is an
evolution of the course ITKA-352:
“Introduction to Watson Technologies”, which aims to provide more
systematic, structured (broader, deeper and multidisciplinary) view to this
popular domain. The major objectives of the course are as follows:
·
To describe challenges and
opportunities within the emerging Cognitive Computing domain and professions
around it;
·
To summarize role and relationships
of Cognitive Computing within the network of closely related scientific
domains, professional fields and courses of the faculty (e.g., Artificial
Intelligence, Semantic and Agent Technologies, Big Data Analytics, Semantic Web
and Linked Data; Cloud Computing, Internet of Things, etc.);
·
To introduce the major providers of
cognitive computing services (Intelligence-as-a-Service) on the market (e.g.,
IBM Watson, Google DeepMind, Microsoft Cognitive Services, etc.) and show demos
of their services (e.g., text, speech, image, sentiment, etc., processing,
analysis, recognition, diagnostics, prediction, etc.);
·
To give introduction on major
theories, methods and algorithms used within cognitive computing services with
particular focus on Deep Learning technology;
·
To provide “friendly” (with
reasonable amount of mathematics) introduction to Deep Learning (including
variations of deep Neural Networks and approaches to train them);
·
To provide different views to this
knowledge suitable to people with different backgrounds and study objectives
(ordinary user, advanced user, software engineer, domain professional, data
scientist, cognitive analyst, mathematician, etc.);
·
To discuss scientific challenges and
open issues within the domain as well as to share with the students information
on relevant ongoing projects in the Faculty;
·
To train within teams to use available
cognitive services via GUIs or APIs for inventing new interesting use cases and
designing own applications;
·
For advanced students there will be a
possibility to contribute (enhance, optimize, etc.) known algorithms or the
related science behind them.
We believe
that knowledge on Cognitive Computing at least at the level of an advanced user
of it would be an excellent added value within the portfolio of every
professional (from very humanitarian to very technical one).
Interesting to notice,
that the major companies provide support to deep machine learning architectures
for cognitive computing with an appropriate hardware. For example, Intel
recently announced a new
chip called Loihi, which is neuromorphic and self-learning chip capable of
representing 130 000 neurons and 130 million synapses. Unlike convolutional neural network (CNN) and other
deep learning processors the Loihi chip uses an
asynchronous spiking model to mimic neuron and synapse behavior in a much
closer analog to animal brain behavior.
Important
challenges around the course topics also include Cybersecurity-related aspects of
the Cognitive Computing, Deep Learning and Collective Intelligence.
Emerging Cognitive Computing services attract huge amounts of users worldwide. Very
recently the new vulnerabilities of Cognitive Computing and of its enabler Deep
Learning have been discovered - the so called Cognitive Risks for Cybersecurity associated with the Cognitive Hack and Data Poisoning attacks. The cyber battleground has shifted recently
from an attack on hard assets to a much softer target: the human mind as human
behavior is the new and last “weakest link” in the cyber security armor. The Bruce
Schneier’s popular quota: “Only amateurs attack machines; professionals target
people” [B. Schneier (2000). Semantic
Attacks: The Third Wave of Network Attacks], is now becoming an
emerging reality. The cognitive hack takes place when a users’
behavior is influenced by misinformation. In his new book [J. Bone (2017). Cognitive
Hack: The New Battleground in Cybersecurity ... the Human Mind, CRC
Press], James Bone admits that “the human-machine interaction is the greatest
threat in cyber space yet very few, if any, security professionals are well
versed in strategies to close this gap”. On the other hand, Cognitive Computing
and Collective Intelligence enhance creation of “artificial” decision-makers,
agents, cognitive robots, etc. These entities are becoming users of various
information systems and sources and automatically learn based on this usage
experiences. Therefore they are also becoming a potential target for Cognitive
Hacking. Very recent articles related to security aspects of Deep Learning
noticed numerous potential risks of influencing outcomes of Machine Learning
(e.g., decision models) by a variety of (training) data poisoning
techniques. Therefore one interesting topic for the self-study would be on how
to handle such risks (based on system's self-awareness
and self-protection) for both human minds and artificial minds (i.e., risks
of Cognitive
Hacking of the Collective Intelligence) to make future smart systems
secure.
We will
combine overview lectures, self-study, group-work, theoretical and practical
assignments and exercises trying to find an optimal approach to everyone.
Recommended reading: Goodfellow,
I., Bengio, Y., & Courville, A. (2016). Deep Learning, MIT Press,
787 pp. (http://www.deeplearningbook.org)
Recommended reading: Michael
Nielsen (2017). Neural
Networks and Deep Learning. (http://neuralnetworksanddeeplearning.com/)
Deep Learning Resources (http://deeplearning.net/)
COMPLETE
PACKAGE OF THE SLIDES FOR THE WHOLE COURSE IS HERE
Lectures for the course TIES4910-Deep Learning for Cognitive
Computing, Theory:
·
All the lectures and self-study material in one PowerPoint. (Lectures
1-10):
o
See the lecture slides
(download before viewing, enable external content and switch on speakers).
o
Contents:
-
Deep Learning & Cognitive
Computing Introduction;
-
Introduction to Neural Networks (for
beginners);
-
Neural Networks and Fuzzy Logic;
-
Strengths and Weaknesses of Deep
Learning;
-
Recurrent Neural Networks;
-
Deep Learning beyond Deep Neural
Networks: What else Can be “Deep”?;
-
Explainable AI and Causality;
-
Deep Learning and Topology;
-
Training Neural Networks
(Backpropagation, Overfitting, Regularization, etc.);
-
Competition on the AI/ML-as-a-Service
Market (perspectives, fears, and concerns);
-
Online Cognitive Services (they can
do almost everything for you);
-
Convolutional Neural Networks;
-
Deconvolution and Style Transfer;
-
Semantic Segmentation;
-
Generative Adversarial Networks and
Generative AI;
-
Introduction to Transfer of Learning
and Federated Learning;
-
Brief Introduction to Self-Supervised
Learning;
-
Attention and Transformers in Deep
Learning;
-
Physics-Informed Neural Networks;
-
Short Introduction to Reinforcement
Learning;
-
Future of Deep Learning (robots,
education, medicine, etc.);
-
Deep Learning and Security:
Adversarial Machine Learning;
-
Few old slides as a summary about IBM
Watson;
-
Become Master in AI and collaborate
with us.
·
Additional old slides.
o
Topic: Introduction to Neural
Networks and Deep Learning
o
See some old additional lecture slides
(download before viewing).
·
Additional old slides:
o
Topic: Convolutional Neural
Networks for Image Processing
o
See some old additional lecture slides
(download before viewing).
·
Additional old slides:
o
Topic: Neural Networks with
Memory: Recurrent Neural Networks and LSTM Networks
o
See some old additional lecture slides
(download before viewing).
Recorded Video-Lectures:
The video records for all
the lectures can be downloaded and viewed from Moodle.
NOTE: if you
have possibility to come to the in-class lectures, choose this option, because
lecture content is constantly updated, and the recorded lectures from previous
years could be slightly outdated …
COMPLETION MODE: Your grade
will be based on the course Assignment (task for the Assignment
is here; deadline: 15 November).
How to continue? We recommend
also taking the course: TIES4911 “Deep Learning for Cognitive
Computing for Developers”.
Collaborate
with our research group on developing stronger AI!