2018/12/12

1

Era of The IntelligenceBIG DATA AND MACHINE INTELLIGENCE

NPU Gr

aduate

Cours

e

by Zha

ngKail

ong

2018/12/12

2

Four Normal Forms for Scientific Research

◦ Experiment Science, describing natural phenomena;◦ 描述自然现象的实验科学

◦ Theory Science, represented by Newton’s Laws and Maxwell’sequations;

◦ 以牛顿定律和麦克斯韦方程等为代表的理论科学

◦ Computation Science, simulating complicated phenomena;◦ 模拟复杂现象的计算科学

◦ Data‐intensive Science◦ 数据密集型科学（2009年微软研究院出版）

What is Machine Intelligence◦ ENIAC, General Mountbatten called “Electronic Brain”;◦ Machine intelligence is truly defined by the founder of computer,Alan Turing, in one of his publications “Computing Machineryand Intelligence”, in 1950.

◦ In fact, this paper didn’t describe how to make a computerintelligent, and also, didn’t put forward any intelligent method forsolving complicated problems, but a verification method to judgeif one computer is intelligent, called Turing Test.◦ Speech/Picture recognition, Machine translate, Human‐MachineConversation, Automatic summary text…

NPU Gr

aduate

Cours

e

by Zha

ngKail

ong

2018/12/12

3

The Faction of Bird Simulation: AI 1.0

◦ In summer of 1956，there was a brainstorming seminar inDartmouth College, called Dartmouth Summer AI Seminar;◦ McCarthy, Minsky, Rochester, Shannon (four are all 29‐year old),and other six scientists;

◦ There’s no new reportable research results. So, they mainlydiscussed some unanswered problems and some ones that werenot studied, such as AI, Natural Language Processing, ANN etc.

◦ The term “Artificial Intelligence” was firstly proposed.◦ This seminar continued almost the whole summer, 1956.

AI 1.0 means the traditional AI method.◦ Opinion: “To obtain intelligence, machineries must think likehuman”;

◦ Firstly, understand how the human intelligence is generated,and then, make the computer think and act according tohuman’s thoughts.—Simulating the human’s or animals’behaviors;

◦ AI 1.0 is very limited!◦ The pen was in the box. The box was in the pen.◦ How to understand? Semantics are not enough to judge, some commonsense and the knowledge about the world and society are also required.

NPU Gr

aduate

Cours

e

by Zha

ngKail

ong

2018/12/12

4

Today, AI owns two meanings◦ In general, referring machine intelligence, that is, any methodthat can make computer pass Turing Test, covering data‐driven,knowledge discover, machine learning, and so on;

◦ In the narrow sense, AI refers to special methods to studymachine intelligence during 1950s and 1960s, such as grammarrules, semantic rules, knowledge inference etc.

DATA

NPU Gr

aduate

Cours

e

by Zha

ngKail

ong

2018/12/12

5

Data is the cornerstone of human civilization.

In recent years, the amount of data is growing almost 40 percent a year.

Data—the cornerstone for establishing human civilization

Obtain Analyze Modeling Predict the future

The key to use data: dependencyThe way of Midas touch: statisticsThe foundation of Data‐driven method: Mathematic Models 

NPU Gr

aduate

Cours

e

by Zha

ngKail

ong

2018/12/12

6

Phenomena Data Information  KnowledgeAn example

◦ “Movements of stars” is a Phenomenon；◦ 自然界的星球运动现象

◦ Through measuring positions and relative times, data can beacquired;

◦ 通过测量星球的位置和对应的时间，得到数据；

◦ Based on these data, the trajectories of stars can be calculated,which is called information;

◦ 通过数据可得到星球运动的轨迹，就是信息；

◦ With aforementioned information, Kepler Laws are summarized,which is knowledge.

◦ 通过信息总结出开普勒三定律，就是知识。

Big Data◦ Big data is a term for data sets that are so large or complexthat traditional data processing application software isinadequate to deal with them. Big data challenges includecapturing data, data storage, data analysis, search, sharing,transfer, visualization, querying, updating and informationprivacy.◦ Volume: big data doesn't sample; it just observes and tracks whathappens

◦ Velocity: big data is often available in real‐time◦ Variety: big data draws from text, images, audio, video; plus itcompletes missing pieces through data fusion;

NPU Gr

aduate

Cours

e

by Zha

ngKail

ong

2018/12/12

7

Big data can be described by the followingcharacteristics◦ Volume◦ The quantity of generated and stored data. The size of the data determinesthe value and potential insight‐ and whether it can actually be considered bigdata or not.

◦ Variety◦ The type and nature of the data. This helps people who analyze it toeffectively use the resulting insight.

◦ Velocity◦ In this context, the speed at which the data is generated and processed tomeet the demands and challenges that lie in the path of growth anddevelopment.

◦ Variability◦ Inconsistency of the data set can hamper processes to handle and manage it.

◦ Veracity◦ The quality of captured data can vary greatly, affecting the accurate analysis.

Applications of Big data derive from real demands, andbenefit from the rapid development of technologies:◦ Technologies and applications, such as broadband internet, mobileinternet, and IoT, produce data continuously;

◦ Computing capability, based on Moore’s Law, is growingthousandfold every ten years;

◦ Could computing decreased the cost of informatization.

◦ More importantly, Machine Intelligence is developing.

NPU Gr

aduate

Cours

e

by Zha

ngKail

ong

2018/12/12

8

The revolution of machine intelligence happened when theamount of accumulated data has reached a singular point toqualitative change.◦ From this point of view, there doesn’t exist essential differencebetween machines’ learning and human being’s learning;

◦ Human knowledges is established with the method of induction,which implies that “the future will be same as the past”, is acontinuity hypothesis;

◦ But, the coming era of intelligence, human will encounter thediscontinuity never met. The degree of revolution brought byArtificial Intelligence will be deeper and wider.

Big data and Machine Intelligence

NPU Gr

aduate

Cours

e

by Zha

ngKail

ong

2018/12/12

9

From 1940s, scientists have contributed to make computersunderstand speech, but never succeeded!

In 1970s, data‐driven methods are employed, which are proved tobe an effective way, but the accuracy rate is not enough;

The accuracy rate is improved at the end of 1990s, when plenty ofdata are accumulated;

After 2000, the amount of data shoot up, for the application ofinternet and mobile internet; The concept of Big data occurs;

Scientists and engineers find that with big data, the intelligence levelof computers have resulted in a leap!

Find new ways: Statistics + Data◦ 1970s，the way “Data‐driven + Supercomputing” was adopted;◦ 1972, during Prof. Frederek Jelinek of Cornell University enjoyed hissabbaticals in IBM, he temporary took charge of the IBM “SmartComputer” project—make computer recognize human voiceautomatically.◦ In previous studies, researchers contributed to make computer simulatehuman’s pronunciation and hearing features (feature extraction) firstly, andthen, employed AI methods (Grammar rules, semantic rules) to understand thewhole sentences; recognition rate 70%, when speak clearly and without noise;

◦ Prof. Jelinek was an expert in the communication domain, rather than intelligence, so, he thought ASR is a communication problem, not an AI problem.

NPU Gr

aduate

Cours

e

by Zha

ngKail

ong

2018/12/12

10

When Jelinek and his members studied ASR, they intentionally created astatistics‐based method to resolve intelligent problem. Since this methoddepends on abundant data, it is called data‐drivenmethod.

◦ 贾里尼克认为：◦ 人脑是信息源，从思考到找到适合的语句，再通过发音说出来，是一个编码的过程，在经过信道传输，听懂是一个解码过程；

◦ 应采用解决通信问题的方法来解决，为此采用了两个数学模型（马尔科夫模型）分别描述信源和信道。

◦ 至于计算机识别是需要从语音中提取什么特征，其想法非常简单，数字通信采用什么特征，语音识别就采用什么特征。

◦ 利用了当时成熟的数字通信技术，彻底抛开了人工智能的那套做法。

◦ 找到数学模型后，接下来就是用统计的方法来训练出模型的参数，也就是今天所讲的机器学习。虽然当时还无互联网，但IBM作为巨头，有大量的电传文本，就成为语音识别系统的最早期数据。

◦ 贾里尼克找到了一个新的方法，同时喜欢招收数学基础好的，特别是学习过理论物理的员工，而把语言学家全部请出了IBM，用了4年时间，开发了一个基于统计方法的语音识别系统，识别率提升指90%。

◦ 随着数据量的积累，系统会变得越来越好，而传统人工智能方法难以受益于数据量的提升。

Automatic Speech Recognition，ASR

科大讯飞最新语音识别系统和框架深度剖析

◦ 科大讯飞研发了一种名为前馈型序列记忆网络FSMN (Feed‐forward Sequential Memory Network) 的新框架。

图4 FSMN结构框图图3 基于RNN——CTC的主流语音识别系统框架

NPU Gr

aduate

Cours

e

by Zha

ngKail

ong

2018/12/12

11

2016, the historic year in the history ofmachine learning.◦ 60 years ago, Maccarthy, Minsky, Rochester, Shannon etc, tenscientists of Dartmouth College put forward the concept of AI;

◦ The last one of them, Minsky, left forever, which marked theend of efforts of the first stage in the field of machine learning;

◦ AlphaGo is the first Robot that defeated the world championGo player, Lee Se‐dol; 2015, only can defeat Fan hui;

◦ Why computers can defeat human?◦ That is mainly because the manner computer obtainsintelligence is different from human’s, it doesn’t dependon logic reasoning, but big data and intelligent algorithms.

◦ Google has employed hundreds of thousands of play dataof Go masters to training AlphaGo.

NPU Gr

aduate

Cours

e

by Zha

ngKail

ong

2018/12/12

12

Two key technologies in AlphaGo◦ Map the status on board to a mathematic model of winning probability, without any artificial rule; Training this model with big data on thousands of servers;

◦ Employ a heuristic search algorithm—Monte Carlo Tree Search, limiting the search space into a preferred scope. Only play the game of Go on several dozens of servers.

The training models and adopted algorithms in AlphaGo areknew decades ago. The efforts of Google is making thesealgorithms running in parallel on hundreds of thousands, andeven, millions of servers!

It promote the capabilities of computers to resolve intelligentproblems constitutionally!

Of course, AlphaGo is in fact not the final goal for Google,developing a machine learning tool is!

NPU Gr

aduate

Cours

e

by Zha

ngKail

ong

2018/12/12

13

Data work wonders: From quantitative changeto qualitative change

1990s, Internet began to rise;

1994‐2004, the error rate of speech recognition isdecreased almost 50%, the accuracy of machinetranslate is doubled.

This is 80% because the increase of data scale, only20% is from the improvement of methods.

2005, the first year of Big data!

Google defeated all teams inthe domain of machinetranslate all over the worldwith huge advantages, withoutany accumulation in thisdomain.

NPU Gr

aduate

Cours

e

by Zha

ngKail

ong

2018/12/12

14

Transfer Intelligent Problems into Data Problems◦ Computer◦ Super computing and storage capabilities◦ The lack of intelligence;◦ Human is not born with intelligence, need study;

◦ Machine intelligence can be obtained by deep learning, thus,the big data problems can be transferred into computableproblems.

◦ Big data and machine intelligence are accompanied by eachother, and this will facilitate the sublimation of Internet fromsensing to knowledge and intelligent decision.

NPU Gr

aduate

Cours

e

by Zha

ngKail

ong