在前两个blog中，已经说了Erlang的历史、应用场景、特点，这次主要演示一个Wordcount的示例，就是给定一个文本文件，统计这个文本文件中的单词以及该单词出现的次数。
今天和群友们讨论了一个问题，突然一下子就上升到哲学角度上了，装逼装大发了。
PS：图片中有错别字，%s/财务和其他9个月/财务和其他9个人/g
不过真心想说的一点是，把Erlang系统，映射到现实中，很多奇葩问题，就能迎刃而解了。所以，在下面的简要设计中，我就尽可能的代入一下现实世界吧。
环境安装
mac 的话，用brew就行了：
brew install erlang
centos 用yum：
yum install erlang
Ubuntu 用apt-get：
apt-get install erlang
源码安装：
$ tar -zxf otp_src_17.4.tar.gz （官网上下载源码）
cd otp_src_17.4
./configure && make -j && make install
注意，确保OpenSSL依赖库已经安装成功，建议安装wxWidgets库。
安装好之后，在控制台terminal下输入"erl"，大概是这样的效果：
简要设计
基本流程设计大概是这样的
1，使用本地计算机的多个终端模拟多个Erlang节点
2，使用一个Erlang进程作为主进程，并且按行读取文本文件（这种读取方式不见得是最好的，后续会细说）
3，主进程按照每行创建一个Erlang进程作为逻辑处理进程
4，逻辑处理进程进行实际的逻辑执行并将处理结果发送给主进程
5，主进程进行汇总并显示一定数量的单词以及该单词出现的次数
|---------------------------------------------------------------------------------------------------------------
| 映射到现实世界中，怎么讲？
| 1，team leader拆分项目需求点
| 2，leader按照每个需求点指派给一个team member
| 3，team member搞定各自的需求点并将结果告知leader
| 4，team leader汇总member的结果并项目结项
|---------------------------------------很自然的现实映射------------------------------------------------------
那那那，注意了，每个逻辑处理进程是可以分布在不同的Erlang节点上的，换言之，如果是在实际的环境中，逻辑处理进程是可以分布在不同的物理机器上。
对于主进程而言，主要做的事情包括：
1，按行读取文本文件
2，创建逻辑处理进程
3，等待逻辑进程的结果返回
4，结果都返回之后，进行汇总并显示最终处理结果
对于逻辑处理进程而言，需要做的事情有：
1，按照指定的字符来切割主进程发送的文本
2，按照单词进行初步的计算和汇总
3，将处理结果发送给主进程
注意注意注意，主进程和逻辑处理进程之间的任务分派，结果传递，都是使用“消息通信的方式”，映射到现实生活中，也很好理解，team leader和team member之间进行沟通就是用说话的方式，人和人之间无法共享彼此的记忆。
简要代码分析
代码分析的目的是为了说明Erlang的方便性，看不太懂代码的同学不用着急，后面的blog会一点一点慢慢解释的。
主体代码就80多行的代码。
首先模块的头部：
-module(wordcount).
-include_lib("stdlib/include/ms_transform.hrl").
-export([start/2]).
-export([worker_execute/3]).
-export([insert_main_ets_table/2]).
-define(BUFSIZE, 1048576).
-define(FILEMODE, [binary, read, {read_ahead, ?BUFSIZE}]).
-define(SPLIT_CHAR_LIST, [<<"\n">>, <<" ">>, <<"\"">>,
<<"!">>, <<"&">>, <<".">>,
<<",">>, <<":">>, <<"--">>]).定义了模块名，导出的函数，一些宏，L11是切割文本的字符。
接着是模块的入口函数，start/2 函数：
start(_, []) ->
"NodeList can not is empty";
start(FileName, NodeList) ->
%% open the file
{ok, FD} = file:open(FileName, ?FILEMODE),
MainEts =
ets:new(mainets, [public,
{write_concurrency, true}]),
TaskOwnerNode = erlang:node(),
TaskResList =
spawn_worker_process(file:read_line(FD), FD, NodeList,
TaskOwnerNode, MainEts, []),
[begin
{WorkerNode, WorkerPid} = task:await(TaskRef),
io:format(" ** One Task return it's result from Node : ~p, worker pid : ~p ~n",
[WorkerNode, WorkerPid]),
ok
end || TaskRef <- TaskResList],
Ms = ets:fun2ms(fun({_, Times} = WordCount) when Times > 10 -> WordCount end),
FinalResult = ets:select(MainEts, Ms),
true = ets:delete(MainEts),
FinalResult.
start/2 函数会先打开指定的文本文件，然后创建一张ets表，用来做“最后的结果汇总”，L11处是调用了另一个函数用来“生成逻辑处理进程”，L13处是等待逻辑处理进程的返回结果，L19是用来做最后汇总并显示一定数量（出现次数大于10次）的单词以及该单词出现的个数。
“生成逻辑处理进程”的函数是这样的：
spawn_worker_process(eof, _FD, _NodeList, _TaskOwnerNode, _MainEts, Res) ->
Res;
spawn_worker_process({ok, Data}, FD, NodeList, TaskOwnerNode, MainEts, Res) ->
Node = get_random_node(NodeList),
TaskRef = task:async(Node, ?MODULE, worker_execute,
[Data, TaskOwnerNode, MainEts]),
spawn_worker_process(file:read_line(FD), FD, NodeList,
TaskOwnerNode, MainEts, [TaskRef | Res]).
使用遍历的方式读取文本文件，然后从“传入的node列表”参数中，随机取一个node，并将该文本作为参数，在node上执行worker_execute函数。
worker_execute函数：
worker_execute(Data, TaskOwnerNode, MainEts) ->
%% <<"this is an example\n">>
%% binary split
[_ | WordList] =
lists:reverse(binary:split(Data, ?SPLIT_CHAR_LIST, [global])),
TempEts = ets:new(tempets, [set]),
ok = lists:foreach(fun(UK) ->
case K = string:to_lower(erlang:binary_to_list(UK)) of
[] ->
ingore;
_ ->
true = etsCountAdd(TempEts, K, {2, 1}, {K, 1})
end
end, WordList),
rpc:call(TaskOwnerNode, ?MODULE,
insert_main_ets_table, [MainEts, ets:tab2list(TempEts)]),
ets:delete(TempEts),
{erlang:node(), erlang:self()}. insert_main_ets_table(MainEts, WorkerResult) ->
[etsCountAdd(MainEts, K, {2, V}, {K, V}) || {K, V} <- WorkerResult],
ok. etsCountAdd(EtsTab, Key, UpdateValue, InsertValue) ->
try ets:insert_new(EtsTab, InsertValue) of
false ->
ets:update_counter(EtsTab, Key, UpdateValue),
true;
_ ->
true
catch _:_ ->
false
end.
L4是按照预先设定的字符切割文本，然后创建一张临时的ets表用来做初步的汇总，然后使用ets update_counter 的方式将{word, times}写入到临时的ets表中，然后通过rpc call 调用主进程所在node 执行insert_main_ets_table 的操作，将逻辑处理进程的处理结果返回给主进程。
至此，代码基本上就是这些。接下来，来执行以下，看看究竟是怎样的效果。
效果演示
先启动4个Erlang节点，名字叫做'[email protected]'，'[email protected]'，'[email protected]'，'[email protected]'，启动之后，大概是这样的。
然后在其中任意一个节点上调用wordcount:start/2 函数。
从上面的红框框中，可以看出，这些逻辑处理进程是随机分布在不同的Erlang节点上的。
然后，我们看看最终的统计结果：
[{"he",},
{"a",},
{"of",},
{"is",},
{"and",},
{"her",},
{"vivienne",},
{"the",},
{"it",},
{"in",},
{"you",},
{"was",},
{"she",},
{"his",},
{"with",},
{"him",},
{"have",},
{"as",},
{"will",},
{"to",},
{"said",},
{"i",},
{"hartley",},
{"be",},
{"that",},
{"at",}]OK，就是这么简单，并发进程就是能这么用，分布式节点就是这么方便。
附
全部的主体代码：
-module(wordcount).
-include_lib("stdlib/include/ms_transform.hrl").
-export([start/2]).
-export([worker_execute/3]).
-export([insert_main_ets_table/2]).
-define(BUFSIZE, 1048576).
-define(FILEMODE, [binary, read, {read_ahead, ?BUFSIZE}]).
-define(SPLIT_CHAR_LIST, [<<"\n">>, <<" ">>, <<"\"">>,
<<"!">>, <<"&">>, <<".">>,
<<",">>, <<":">>, <<"--">>]).
start(_, []) ->
"NodeList can not is empty";
start(FileName, NodeList) ->
%% open the file
{ok, FD} = file:open(FileName, ?FILEMODE),
MainEts =
ets:new(mainets, [public,
{write_concurrency, true}]),
TaskOwnerNode = erlang:node(),
TaskResList =
spawn_worker_process(file:read_line(FD), FD, NodeList,
TaskOwnerNode, MainEts, []),
[begin
{WorkerNode, WorkerPid} = task:await(TaskRef),
io:format(" ** One Task return it's result from Node : ~p, worker pid : ~p ~n",
[WorkerNode, WorkerPid]),
ok
end || TaskRef <- TaskResList],
Ms = ets:fun2ms(fun({_, Times} = WordCount) when Times > 10 -> WordCount end),
FinalResult = ets:select(MainEts, Ms),
true = ets:delete(MainEts),
FinalResult.
spawn_worker_process(eof, _FD, _NodeList, _TaskOwnerNode, _MainEts, Res) ->
Res;
spawn_worker_process({ok, Data}, FD, NodeList, TaskOwnerNode, MainEts, Res) ->
Node = get_random_node(NodeList),
TaskRef = task:async(Node, ?MODULE, worker_execute,
[Data, TaskOwnerNode, MainEts]),
spawn_worker_process(file:read_line(FD), FD, NodeList,
TaskOwnerNode, MainEts, [TaskRef | Res]).
get_random_node(NodeList) ->
NodeListLen = erlang:length(NodeList),
lists:nth(get_random(NodeListLen), NodeList).
get_random(Num) ->
{Res, _} = random:uniform_s(Num, erlang:now()),
Res.
worker_execute(Data, TaskOwnerNode, MainEts) ->
%% <<"this is an example\n">>
%% binary split
[_ | WordList] =
lists:reverse(binary:split(Data, ?SPLIT_CHAR_LIST, [global])),
TempEts = ets:new(tempets, [set]),
ok = lists:foreach(fun(UK) ->
case K = string:to_lower(erlang:binary_to_list(UK)) of
[] ->
ingore;
_ ->
true = etsCountAdd(TempEts, K, {2, 1}, {K, 1})
end
end, WordList),
rpc:call(TaskOwnerNode, ?MODULE,
insert_main_ets_table, [MainEts, ets:tab2list(TempEts)]),
ets:delete(TempEts),
{erlang:node(), erlang:self()}.
insert_main_ets_table(MainEts, WorkerResult) ->
[etsCountAdd(MainEts, K, {2, V}, {K, V}) || {K, V} <- WorkerResult],
ok.
etsCountAdd(EtsTab, Key, UpdateValue, InsertValue) ->
try ets:insert_new(EtsTab, InsertValue) of
false ->
ets:update_counter(EtsTab, Key, UpdateValue),
true;
_ ->
true
catch _:_ ->
false
end.说明，其中的一个task模块，是我自己封装的，在这：https://github.com/redink-toys/task
文本文件：
IN GILT letters on the ground glass of the door of room No. were the words: "Robbins & Hartley, Brokers." The clerks had gone. It was past five, and with the solid tramp of a drove of prize Percherons, scrub- women were invading the cloud-capped twenty-story office building. A puff of red-hot air flavoured with lemon peelings, soft-coal smoke and train oil came in through the half-open windows.
Robbins, fifty, something of an overweight beau, and addicted to first nights and hotel palm-rooms, pretended to be envious of his partner's commuter's joys.
"Going to be something doing in the humidity line to-night," he said. "You out-of-town chaps will be the people, with your katydids and moonlight and long drinks and things out on the front porch."
Hartley, twenty-nine, serious, thin, good-looking, ner- vous, sighed and frowned a little.
"Yes," said he, "we always have cool nights in Floral- hurst, especially in the winter."
A man with an air of mystery came in the door and went up to Hartley.
"I've found where she lives," he announced in the portentous half-whisper that makes the detective at work a marked being to his fellow men.
Hartley scowled him into a state of dramatic silence and quietude. But by that time Robbins had got his cane and set his tie pin to his liking, and with a debonair nod went out to his metropolitan amusements.
"Here is the address," said the detective in a natural tone, being deprived of an audience to foil.
Hartley took the leaf torn out of the sleuth's dingy memorandum book. On it were pencilled the words "Vivienne Arlington, No. 341 East --th Street, care of Mrs.
McComus."
"Moved there a week ago," said the detective. "Now, if you want any shadowing done, Mr. Hartley, I can do you as fine a job in that line as anybody in the city. It will be only $7 a day and expenses. Can send in a daily typewritten report, covering -- "
"You needn't go on," interrupted the broker. "It isn't a case of that kind. I merely wanted the address. How much shall I pay you?"
"One day's work," said the sleuth. "A tenner will cover it."
Hartley paid the man and dismissed him. Then he left the office and boarded a Broadway car. At the first large crosstown artery of travel he took an eastbound car that deposited him in a decaying avenue, whose ancient structures once sheltered the pride and glory of the town.
Walking a few squares, he came to the building that he sought. It was a new flathouse, bearing carved upon its cheap stone portal its sonorous name, "The Vallambrosa."
Fire-escapes zigzagged down its front -- these laden with household goods, drying clothes, and squalling children evicted by the midsummer heat. Here and there a pale rubber plant peeped from the miscellaneous mass, as if wondering to what kingdom it belonged -- vegetable, animal or artificial.
Hartley pressed the "McComus" button. The door latch clicked spasmodically -- now hospitably, now doubt- fully, as though in anxiety whether it might be admitting friends or duns. Hartley entered and began to climb the stairs after the manner of those who seek their friends in city flat-houses -- which is the manner of a boy who climbs an apple-tree, stopping when he comes upon what he wants.
On the fourth floor he saw Vivienne standing in an open door. She invited him inside, with a nod and a bright, genuine smile. She placed a chair for him near a window, and poised herself gracefully upon the edge of one of those Jekyll-and-Hyde pieces of furniture that are masked and mysteriously hooded, unguessable bulks by day and inquisitorial racks of torture by night.
Hartley cast a quick, critical, appreciative glance at her before speaking, and told himself that his taste in choosing had been flawless.
Vivienne was about twenty-one. She was of the purest Saxon type. Her hair was a ruddy golden, each filament of the neatly gathered mass shining with its own lustre and delicate graduation of colour. In perfect harmony were her ivory-clear complexion and deep sea-blue eyes that looked upon the world with the ingenuous calmness of a mermaid or the pixie of an undiscovered mountain stream. Her frame was strong and yet possessed the grace of absolute naturalness. And yet with all her North- ern clearness and frankness of line and colouring, there seemed to be something of the tropics in her -- something of languor in the droop of her pose, of love of ease in her ingenious complacency of satisfaction and comfort in the mere act of breathing -- something that seemed to claim for her a right as a perfect work of nature to exist and be admired equally with a rare flower or some beauti- ful, milk-white dove among its sober-hued companions.
She was dressed in a white waist and dark skirt - that discreet masquerade of goose-girl and duchess.
"Vivienne," said Hartley, looking at her pleadingly, "you did not answer my last letter. It was only by nearly a week's search that I found where you had moved to. Why have you kept me in suspense when you knew how anxiously I was waiting to see you and hear from you?"
The girl looked out the window dreamily.
"Mr. Hartley," she said hesitatingly, "I hardly know what to say to you. I realize all the advantages of your offer, and sometimes I feel sure that I could be contented with you. But, again, I am doubtful. I was born a city girl, and I am afraid to bind myself to a quiet sub- urban life."
"My dear girl," said Hartley, ardently, "have I not told you that you shall have everything that your heart can desire that is in my power to give you? You shall come to the city for the theatres, for shopping and to visit your friends as often as you care to. You can trust me, can you not?"
"To the fullest," she said, turning her frank eyes upon him with a smile. "I know you are the kindest of men, and that the girl you get will be a lucky one. I learned all about you when I was at the Montgomerys'."
"Ah!" exclaimed Hartley, with a tender, reminiscent light in his eye; "I remember well the evening I first saw you at the Montgomerys'. Mrs. Montgomery was sound- ing your praises to me all the evening. And she hardly did you justice. I shall never forget that supper. Come, Vivienne, promise me. I want you. You'll never regret coming with me. No one else will ever give you as pleasant a home."
The girl sighed and looked down at her folded hands.
A sudden jealous suspicion seized Hartley.
"Tell me, Vivienne," he asked, regarding her keenly, "is there another -- is there some one else ?"
A rosy flush crept slowly over her fair cheeks and neck.
"You shouldn't ask that, Mr. Hartley," she said, in some confusion. "But I will tell you. There is one other -- but he has no right -- I have promised him nothing."
"His name?" demanded Hartley, sternly.
"Townsend."
"Rafford Townsend!" exclaimed Hartley, with a grim tightening of his jaw. "How did that man come to know you? After all I've done for him -- "
"His auto has just stopped below," said Vivienne, bending over the window-sill. "He's coming for his answer. Oh I don't know what to do!"
The bell in the flat kitchen whirred. Vivienne hurried to press the latch button.
"Stay here," said Hartley. "I will meet him in the hall."
Townsend, looking like a Spanish grandee in his light tweeds, Panama hat and curling black mustache, came up the stairs three at a time. He stopped at sight of Hartley and looked foolish.
"Go back," said Hartley, firmly, pointing downstairs with his forefinger.
"Hullo!" said Townsend, feigning surprise. "What's up? What are you doing here, old man?"
"Go back," repeated Hartley, inflexibly. "The Law of the Jungle. Do you want the Pack to tear you in pieces? The kill is mine."
"I came here to see a plumber about the bathroom connections," said Townsend, bravely.
"All right," said Hartley. "You shall have that lying plaster to stick upon your traitorous soul. But, go back." Townsend went downstairs, leaving a bitter word to be wafted up the draught of the staircase. Hartley went back to his wooing.
"Vivienne," said he, masterfully. "I have got to have you. I will take no more refusals or dilly-dallying."
"When do you want me?" she asked.
"Now. As soon as you can get ready."
She stood calmly before him and looked him in the eye.
"Do you think for one moment," she said, "that I would enter your home while Héloise is there?"
Hartley cringed as if from an unexpected blow. He folded his arms and paced the carpet once or twice.
"She shall go," he declared grimly. Drops stood upon his brow. "Why should I let that woman make my life miserable? Never have I seen one day of freedom from trouble since I have known her. You are right, Vivienne. Héloise must be sent away before I can take you home. But she shall go. I have decided. I will turn her from my doors."
"When will you do this?" asked the girl.
Hartley clinched his teeth and bent his brows together.
"To-night," he said, resolutely. "I will send her away to-night."
"Then," said Vivienne, "my answer is 'yes.' Come for me when you will."
She looked into his eyes with a sweet, sincere light in her own. Hartley could scarcely believe that her sur- render was true, it was so swift and complete.
"Promise me," he said feelingly, "on your word and honour."
"On my word and honour," repeated Vivienne, softly.
At the door he turned and gazed at her happily, but yet as one who scarcely trusts the foundations of his joy.
"To-morrow," he said, with a forefinger of reminder uplifted.
"To-morrow," she repeated with a smile of truth and candour.
In an hour and forty minutes Hartley stepped off the train at Floralhurst. A brisk walk of ten minutes brought him to the gate of a handsome two-story cottage set upon a wide and well-tended lawn. Halfway to the house he was met by a woman with jet-black braided hair and flowing white summer gown, who half strangled him without apparent cause.
When they stepped into the hall she said:
"Mamma's here. The auto is coming for her in half an hour. She came to dinner, but there's no dinner."
"I've something to tell you," said Hartley. "I thought to break it to you gently, but since your mother is here we may as well out with it."
He stooped and whispered something at her ear.
His wife screamed. Her mother came running into the hall. The dark-haired woman screamed again- the joyful scream of a well-beloved and petted woman.
"Oh, mamma!" she cried ecstatically, "what do you think? Vivienne is coming to cook for us! She is the one that stayed with the Montgomerys a whole year. And now, Billy, dear," she concluded, "you must go right down into the kitchen and discharge Héloise. She has been drunk again the whole day long."
总结
并发进程
分布式
对了，上面有提到，“按行读取文本文件”不见得是最好的方式。如果一个文本文件中有很多行，但是每行的文本又非常小，就有可能出现问题。本blog的示例，主要是为了演示Erlang编程语言在并发进程和分布式特点上的优势。在实际的场景中，断然需要做一些结构上的调整。欢迎讨论。
“Erlang基础 -- 介绍”这次blog应该就是一个换行符，下一次主要开始shell系统、变量和常见的数据类型。