I have an application runs perfectly on single machine, it was designed to be used this way, but things got changed, now, at least three copies have deployed, and is continue increasing.
The main job of the application is to collect data, and then do analysis, being used by three products means deploy it to three machines.
It always runs background with three processes, so I need to keep an eye on every processes, trace logs and set nearly the same configurations to each copy on every machine, as well as some product specific settings. It’s chaos.
I plan to make it distributed, with one server do the product specific analysis, and some clients collecting data. Server dispatches clients, gives clients job to do, sets configurations per client, and gets running status of all clients.
Hope this can relieve me. :)
I pick Thrift as my RPC framework, tried it, it’s easy and fun except the not-so-good documentation.
First, create .thrift file.
1 2 3 4 5 6 7 8 9 10 11 12
Types and services are defined in this file, I define a type
Item, and a service named
CrawlingService, which has one method
write, it will be used as remote method.
Then, use thrift compiler to compile the file.
The compiler must be installed, I use Mac, which has the amazing Homebrew, “brew install thrift”, done.
thrift -out . --gen java item.thrift to compile.
-out . flag to not create the gen-java directory. If not, it will create java files in
gen-java/me/zhengdong/thrift, which is not what I need.
Two java files are generated,
Implement the service handler
CrawlingService has a
write method, we need to implement it.
1 2 3 4 5 6 7 8
This will be used in RPC, client can call this method by putting items it collects, then server will print all the items out.
Finally, make the server and client
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
We can set the IP which server runs on to replace the “localhost”, then, every invoke of
Client.write on any machine will make server prints.
You can see the tutorial here, I feel it’s funny to call it “tutorial”.
If you want to learn more about Thrift, please read the white paper.
The tutorial link above no longer exists, then I google and find the official tutorial looks better now.