I N T R O T O N O D E . J S
- L E V E L O N E -
INTRO TO NODE.JS
WHAT IS NODE.JS?
It’s fast because it’s mostly C code
Allows you to build scalable network
applications using JavaScript on the server-side.
V8 JavaScript Runtime
Node.js
INTRO TO NODE.JS
WHAT COULD YOU BUILD?
• Websocket Server
• Fast File Upload Client
• Ad Server
• Any Real-Time Data Apps
Like a chat server
INTRO TO NODE.JS
WHAT IS NODE.JS NOT ?
• A Web Framework
• For Beginners
It’s very low level
• Multi-threaded
You can think of it as a single threaded server


INTRO TO NODE.JS
OBJECTIVE: PRINT FILE CONTENTS
This is a “Callback”
Read file from Filesystem, set equal to “contents”
Print contents
• Blocking Code
• Non-Blocking Code
Do something else
Read file from Filesystem
whenever you’re complete, print the contents
Do Something else

console.log(contents);
INTRO TO NODE.JS
BLOCKING VS NON-BLOCKING
var contents = fs.readFileSync('/etc/hosts');
console.log(contents);
console.log('Doing something else');
• Blocking Code
• Non-Blocking Code
console.log('Doing something else');
Stop process until
complete
fs.readFile('/etc/hosts', function(err, contents) {
});
fs.readFile('/etc/hosts', function(err, contents) {
console.log(contents);
});
INTRO TO NODE.JS
CALLBACK ALTERNATE SYNTAX
var callback = function(err, contents) {
console.log(contents);
}
fs.readFile('/etc/hosts', callback);
Same as
INTRO TO NODE.JS
BLOCKING VS NON-BLOCKING
blocking
0s
non-blocking
10s
5s
0s
10s
5s
fs.readFile('/etc/hosts', callback);
fs.readFile('/etc/inetcfg', callback);
var callback = function(err, contents) {
console.log(contents);
}
hello.js


NODE.JS HELLO DOG
$ curl http://localhost:8080
Hello, this is dog.
How we require modules
Status code in header
Response body
Close the connection
Listen for connections on this port

$ node hello.js Run the server
var http = require('http');
http.createServer(function(request, response) {
response.writeHead(200);
response.write("Hello, this is dog.");
response.end();
}).listen(8080);
console.log('Listening on port 8080...');
Listening on port 8080...
THE EVENT LOOP
var http = require('http');
http.createServer(function(request, response) {
}).listen(8080);
console.log('Listening on port 8080...');
Starts the Event Loop when finished
...
Known Events
request
Checking
for
Events
Run the Callback
INTRO TO NODE.JS
WHY JAVASCRIPT ?
“JavaScript has certain characteristics that make it very
different than other dynamic languages, namely that it has
no concept of threads. Its model of concurrency is
completely based around events.”
- Ryan Dahl
THE EVENT LOOP
Known Events
request
Checking
for
Events
connection
close
Event Queue
close
request
Events processed one at a time

INTRO TO NODE.JS
WITH LONG RUNNING PROCESS
Represent long running process
var http = require('http');
http.createServer(function(request, response) {
}).listen(8080);
response.writeHead(200);
response.write("Dog is done.");
response.end();
setTimeout(function(){
}, 5000); 5000ms = 5 seconds
response.write("Dog is running.");

INTRO TO NODE.JS
TWO CALLBACKS HERE
var http = require('http');
http.createServer(function(request, response) {
response.writeHead(200);
request
timeout
}).listen(8080);
response.write("Dog is done.");
response.end();
setTimeout(function(){
}, 5000);
response.write("Dog is running.");
TWO CALLBACKS TIMELINE
0s
10s
5s
Request comes in, triggers request event
Request Callback executes
setTimeout registered
Request comes in, triggers request event
Request Callback executes
setTimeout registered
triggers setTimeout event
setTimeout Callback executes
triggers setTimeout event
setTimeout Callback
request
timeout
WITH BLOCKING TIMELINE
0s
10s
5s
Request comes in, triggers request event
Request Callback executes
setTimeout executed
Request comes in, waits for server
Request comes in
triggers setTimeout event
setTimeout Callback executed
Wasted Time
Request Callback executes
INTRO TO NODE.JS
• Calls out to web services
TYPICAL BLOCKING THINGS
• Reads/Writes on the Database
• Calls to extensions
E V E N T S
- L E V E L T W O -
EVENTS
EVENTS IN THE DOM
The DOM
The DOM triggers Events
click
events
you can listen for those events
submit
hover
When ‘click’ event is triggered
attach
$("p").on("click", function(){ ... });
EVENTS
EVENTS IN NODE
EventEmitter
Many objects in Node emit events
net.Server
request
event
EventEmitter
fs.readStream
data
event
EVENTS
CUSTOM EVENT EMITTERS



var logger = new EventEmitter();
logger.emit('error', 'Spilled Milk');
ERR: Spilled Milk

logger.emit('error', 'Eggs Cracked');

var EventEmitter = require('events').EventEmitter;
error
warn
info
listen for error event
logger.on('error', function(message){
console.log('ERR: ' + message);
});
ERR: Eggs Cracked
events
EVENTS
EVENTS IN NODE
EventEmitter
Many objects in Node emit events
net.Server
request
event
When ‘request’ event is emitted
function(request, response){ .. }
emit
attach
EVENTS
HTTP ECHO SERVER
http.createServer(function(request, response){ ... });
But what is really going on here?
http://nodejs.org/api/
EVENTS
BREAKING IT DOWN
http.createServer(function(request, response){ ... });
http.createServer(function(request, response){ ... });
EVENTS
ALTERNATE SYNTAX
var server = http.createServer();
function(request, response){ ... });
server.on('request',
This is how we add
Same as
function(){ ... });
server.on('close',
add event listeners
S T R E A M S
- L E V E L T H R E E -
STREAMS
WHAT ARE STREAMS?
Start Processing
Immediately
Streams can be readable, writeable, or both

STREAMS
STREAMING RESPONSE
Our clients receive
"Dog is running."
"Dog is done."
(5 seconds later)
http.createServer(function(request, response) {
}).listen(8080);
response.writeHead(200);
response.write("Dog is done.");
response.end();
setTimeout(function(){
}, 5000);
response.write("Dog is running.");
readable stream
writable stream
STREAMS
HOW TO READ FROM THE REQUEST ?
EventEmitter
Readable Stream
data
events
emit
Lets print what we receive from the request.
http.createServer(function(request, response) {
request.on('data', function(chunk) {
console.log(chunk.toString());
});
response.end();
}).listen(8080)
request.on('end', function() {
});
response.writeHead(200);
end
STREAMS
LETS CREATE AN ECHO SERVER
http.createServer(function(request, response) {
request.on('data', function(chunk) {
});
response.end();
}).listen(8080)
request.on('end', function() {
});
response.writeHead(200);
response.write(chunk);
request.pipe(response);
STREAMS
LETS CREATE AN ECHO SERVER!
http.createServer(function(request, response) {
}).listen(8080)
request.pipe(response);

$ curl -d 'hello' http://localhost:8080
Hello on client
response.writeHead(200);
cat 'bleh.txt' | grep 'something'
Kinda like on the command line
STREAMS
READING AND WRITING A FILE
var fs = require('fs');
var file = fs.createReadStream("readme.md");
var newFile = fs.createWriteStream("readme_copy.md");
require filesystem module
file.pipe(newFile);
STREAMS
UPLOAD A FILE
var fs = require('fs');
var newFile = fs.createWriteStream("readme_copy.md");
var http = require('http');
http.createServer(function(request, response) {
request.pipe(newFile);
}).listen(8080);

$ curl --upload-file readme.md http://localhost:8080
uploaded!
response.end('uploaded!');
request.on('end', function() {
});
THE AWESOME STREAMING
client
storage
server
original file
transferred file
non-blocking
0s
10s
5s
BACK PRESSURE!
client
storage
server
original file
transferred file
Writable stream slower
than readable stream
Using pipe solves this problem
THINK OF A MILK JUG
milkStream.pause();
milkStream.resume();
});
Once milk jug is drained


PIPE SOLVES BACKPRESSURE
readStream.resume();
});
Pause when writeStream is full
writeStream.on('drain', function(){
readStream.on('data', function(chunk) {
writeStream.write(chunk);
});
var buffer_good =
if (!buffer_good) readStream.pause();
returns false
if kernel buffer full
Resume when ready to write again
readStream.pipe(writeStream);
All encapsulated in
STREAMS
FILE UPLOADING PROGRESS

STREAMS
FILE UPLOADING PROGRESS
$ curl --upload-file file.jpg http://localhost:8080

progress: 3%
progress: 6%
progress: 9%
progress: 12%
progress: 13%
...
progress: 99%
progress: 100%
Outputs:
• HTTP Server
• File System
We’re going to need:
STREAMS
DOCUMENTATION http://nodejs.org/api/
Stability Scores
STREAMS
REMEMBER THIS CODE?
var fs = require('fs');
var newFile = fs.createWriteStream("readme_copy.md");
var http = require('http');
http.createServer(function(request, response) {
request.pipe(newFile);
}).listen(8080);
response.end('uploaded!');
request.on('end', function() {
});
STREAMS
REMEMBER THIS CODE?
var newFile = fs.createWriteStream("readme_copy.md");
http.createServer(function(request, response) {
request.pipe(newFile);
}).listen(8080);
...
request.on('data', function(chunk) {
uploadedBytes += chunk.length;
var progress = (uploadedBytes / fileBytes) * 100;
response.write("progress: " + parseInt(progress, 10) + "%\n");
});
var uploadedBytes = 0;
var fileBytes = request.headers['content-length'];
STREAMS
SHOWING PROGRESS
M O D U L E S
- L E V E L F O U R -

MODULES
REQUIRING MODULES
http.js
How does it find these files?
var http = require('http');

var fs = require('fs');
fs.js
How does ‘require’ return the libraries?

MODULES
LETS CREATE OUR OWN MODULE

custom_hello.js

custom_goodbye.js
app.js
exports = hello;
var hello = require('./custom_hello');
hello();
exports defines what require returns
var hello = function() {
console.log("hello!");
}
exports.goodbye = function() {
console.log("bye!");
}
var gb = require('./custom_goodbye');
gb.goodbye();

require('./custom_goodbye').goodbye();
If we only need to call once

MODULES
EXPORT MULTIPLE FUNCTIONS

my_module.js
app.js
var foo = function() { ... }
var bar = function() { ... }
exports.foo = foo
exports.bar = bar
var myMod = require('./my_module');
myMod.foo();
myMod.bar();
my_module.js
foo
bar
var baz = function() { ... }
baz
“private”
MODULES
MAKING HTTP REQUESTS
app.js
logs response body
begins request
var http = require('http');
var options = {
host: 'localhost', port: 8080, path: '/', method: 'POST'
}
var request = http.request(options, function(response){
response.on('data', function(data){
console.log(data);
});
});
request.end();
request.write(message);
finishes request
var message = "Here's looking at you, kid.";

MODULES
ENCAPSULATING THE FUNCTION
app.js
var http = require('http');
var makeRequest = function(message) {
var options = {
host: 'localhost', port: 8080, path: '/', method: 'POST'
}
var request = http.request(options, function(response){
response.on('data', function(data){
console.log(data);
});
});
request.end();
}
makeRequest("Here's looking at you, kid.");
request.write(message);
Text
MODULES
CREATING & USING A MODULE
make_request.js
var http = require('http');
var makeRequest = function(message) {
}
exports = makeRequest;
...

app.js
var makeRequest = require('./make_request');
makeRequest("Here's looking at you, kid");
makeRequest("Hello, this is dog");
Where does require look for modules?


MODULES
REQUIRE SEARCH
var make_request = require('make_request')
/Home/eric/my_app/app.js
• /Home/eric/my_app/node_modules/
var make_request = require('./make_request')
var make_request = require('../make_request')
var make_request = require('/Users/eric/nodes/make_request')
• /Home/eric/node_modules/make_request.js
• /Home/node_modules/make_request.js
• /node_modules/make_request.js
look in same directory
look in parent directory
Search in node_modules directories
MODULES
NPM: THE USERLAND SEA
Package manager for node
• Comes with node
• Module Repository
• Dependency Management
• Easily publish modules
• “Local Only”
“Core” is small. “Userland” is large.
MODULES
INSTALLING A NPM MODULE

$ npm install request
https://github.com/mikeal/request
In /Home/my_app
Home
my_app
request
node_modules
Installs into local node_modules directory

var request = require('request');
In /Home/my_app/app.js
Loads from local node_modules directory


MODULES
LOCAL VS GLOBAL
Global npm modules can’t be required

$ npm install coffee-script -g
Install modules with executables globally
$ coffee app.coffee
var coffee = require('coffee-script');

$ npm install coffee-script

var coffee = require('coffee-script');
global
Install them locally
MODULES
FINDING MODULES
npm registry

$ npm search request
npm command line
github search
toolbox.no.de
MODULES
DEFINING YOUR DEPENDENCIES
my_app/package.json
version number

$ npm install
my_app
connect
node_modules
Installs into the node_modules directory
{
"name": "My App",
"version": "1",
"dependencies": {
"connect": "1.8.7"
}
}
MODULES
DEPENDENCIES
my_app
connect
node_modules
Installs sub-dependencies
connect
node_modules
qs
connect
node_modules
mime
connect
node_modules
formidable
No conflicting modules!
"dependencies": {
"connect": "1.8.7"
}
my_app/package.json



MODULES
SEMANTIC VERSIONING
"connect": "1.8.7"
1
8
7
Major Minor
Patch
.
.
http://semver.org/

"connect": "~1.8.7"
>=1.8.7 <1.9.0
Considered safe
"connect": "~1.8"
>=1.8 <2.0.0
API could change
"connect": "~1"
>=1.0.0 <2.0.0
Dangerous
Ranges
E X P R E S S
- L E V E L F I V E -
EXPRESS
EXPRESS
“Sinatra inspired web development framework for Node.js --
insanely fast, flexible, and simple”
• Easy route URLs to callbacks
• Middleware (from Connect)
• Environment based configuration
• Redirection helpers
• File Uploads




EXPRESS
INTRODUCING EXPRESS
$ npm install express
var express = require('express');

var app = express.createServer();
app.get('/', function(request, response) {
response.sendfile(__dirname + "/index.html");
});
app.listen(8080);

$ curl http://localhost:8080/
> 200 OK
root route
current directory

EXPRESS
EXPRESS ROUTES
app.js
route definition
get the last 10 tweets for screen_name
pipe the request to response
var request = require('request');
var url = require('url');
app.get('/tweets/:username', function(req, response) {
var username = req.params.username;
options = {
protocol: "http:",
host: 'api.twitter.com',
pathname: '/1/statuses/user_timeline.json',
query: { screen_name: username, count: 10}
}
var twitterUrl = url.format(options);
request(twitterUrl).pipe(response);
});
EXPRESS
EXPRESS ROUTES
EXPRESS
EXPRESS + HTML


EXPRESS
EXPRESS TEMPLATES
app.js
tweets.ejs
app.get('/tweets/:username', function(req, response) {
...
request(url, function(err, res, body) {
var tweets = JSON.parse(body);
response.render('tweets.ejs', {tweets: tweets, name: username});
});
});

Tweets for @<%= name %>

<% tweets.forEach(function(tweet){ %>

• <%= tweet.text %>

<% }); %>

EXPRESS
EXPRESS TEMPLATES

EXPRESS
TEMPLATE LAYOUTS






<%- body %>

Tweets for @<%= name %>

<% tweets.forEach(function(tweet){ %>

• <%= tweet.text %>

<% }); %>

tweets.ejs
layout.ejs
EXPRESS
EXPRESS TEMPLATES
S O C K E T . I O
- L E V E L S I X -
SOCKET.IO
CHATTR
SOCKET.IO
WEBSOCKETS
browser
traditional server
Traditional request/response cycle
Using duplexed websocket connection
S O C K E T . I O
WEBSOCKETS
browser
socket.io
SOCKET.IO
SOCKET.IO FOR WEBSOCKETS
var socket = require('socket.io');
var app = express.createServer();
var io = socket.listen(app);
Abstracts websockets with fallbacks
io.sockets.on('connection', function(client) {
});
console.log('Client connected...');

var server = io.connect('http://localhost:8080');


$ npm install socket.io
app.js
index.html
SOCKET.IO
SENDING MESSAGES TO CLIENT
io.sockets.on('connection', function(client) {
});
console.log('Client connected...');

var server = io.connect('http://localhost:8080');

app.js
index.html
client.emit('messages', { hello: 'world' });
server.on('messages', function (data) {
});
alert(data.hello);
emit the ‘messages’ event on the client
listen for ‘messages’ events
SOCKET.IO
CHATTR HELLO WORLD
SOCKET.IO
SENDING MESSAGES TO SERVER
io.sockets.on('connection', function(client) {
});
var server = io.connect('http://localhost:8080');

app.js
index.html
client.on('messages', function (data) {
});
console.log(data);
$('#chat_form').submit(function(e){
var message = $('#chat_input').val();
socket.emit('messages', message);
});
listen for ‘messages’ events
emit the ‘messages’ event on the server
SOCKET.IO
CHATTR HELLO WORLD
SOCKET.IO
BROADCASTING MESSAGES

clients
server
app.js
socket.broadcast.emit("message", 'Hello');
SOCKET.IO
BROADCASTING MESSAGES
io.sockets.on('connection', function(client) {
});

app.js
index.html
client.on('messages', function (data) {
});
...
broadcast message to all other clients connected
client.broadcast.emit("messages", data);
server.on('messages', function(data) { insertMessage(data) });
insert message into the chat
SOCKET.IO
BROADCASTING MESSAGES
SOCKET.IO
SAVING DATA ON THE SOCKET
io.sockets.on('connection', function(client) {
});
var server = io.connect('http://localhost:8080');

app.js
index.html
client.on('join', function(name) {
client.set('nickname', name);
});
set the nickname associated
with this client
server.on('connect', function(data) {
$('#status').html('Connected to chattr');
nickname = prompt("What is your nickname?");
server.emit('join', nickname);
});
notify the server of the
users nickname
SOCKET.IO
SAVING DATA ON THE CLIENT
io.sockets.on('connection', function(client) {
});
app.js
client.on('join', function(name) {
client.set('nickname', name);
});
set the nickname associated
with this client
client.on('messages', function(data){
});
client.broadcast.emit("chat", name + ": " + message);
client.get('nickname', function(err, name) {
});
get the nickname of this client before broadcasting message
broadcast with the name and message
SOCKET.IO
SAVING DATA ON THE CLIENT
P E R S I S T I N G D A T A
- L E V E L S E V E N -
PERSISTING DATA
RECENT MESSAGES
});
});
});
PERSISTING DATA
RECENT MESSAGES
io.sockets.on('connection', function(client) {
client.on('join', function(name) {
client.set('nickname', name);
client.broadcast.emit("chat", name + " joined the chat");
});
client.on("messages", function(message){
client.get("nickname", function(error, name) {
client.broadcast.emit("messages", name + ": " + message);
app.js
});
});
});
PERSISTING DATA
STORING MESSAGES
io.sockets.on('connection', function(client) {
client.on("messages", function(message){
client.get("nickname", function(error, name) {
app.js
storeMessage(name, message);
var messages = []; store messages in array
var storeMessage = function(name, data){
messages.push({name: name, data: data});
if (messages.length > 10) {
messages.shift();
}
}
add message to end of array
if more than 10 messages long,
remove the last one
when client sends a message
call storeMessage
PERSISTING DATA
EMITTING MESSAGES
io.sockets.on('connection', function(client) {
});
app.js
client.on('join', function(name) {
});
messages.forEach(function(message) {
client.emit("messages", message.name + ": " + message.data);
});
iterate through messages array
and emit a message on the connecting
client for each one
...
PERSISTING DATA
RECENT MESSAGES
PERSISTING DATA
PERSISTING STORES
• MongoDB
• CouchDB
• PostgreSQL
• Memcached
• Riak
All non-blocking!
Redis is a key-value store
PERSISTING DATA
REDIS DATA STRUCTURES
Strings
SET, GET, APPEND, DECR, INCR...
Hashes
HSET, HGET, HDEL, HGETALL...
Lists
LPUSH, LREM, LTRIM, RPOP, LINSERT...
Sets
SADD, SREM, SMOVE, SMEMBERS...
Sorted Sets
ZADD, ZREM, ZSCORE, ZRANK...
data structure
commands
PERSISTING DATA
REDIS COMMAND DOCUMENTATION
PERSISTING DATA
NODE REDIS

client.get("message1", function(err, reply){
console.log(reply);
});

PERSISTING DATA
REDIS
key
value
"hello, yes this is dog"
var redis = require('redis');
var client = redis.createClient();
client.set("message1", "hello, yes this is dog");
client.set("message2", "hello, no this is spider");
commands are non-blocking

$ npm install redis
PERSISTING DATA
REDIS LISTS: PUSHING
Add a string to the “messages” list

client.lpush("messages", message, function(err, reply){
console.log(reply);
});
"1”

var message = "Hello, no this is spider";
client.lpush("messages", message, function(err, reply){
console.log(reply);
});
"2”
replies with list length
Add another string to “messages”
var message = "Hello, this is dog";

PERSISTING DATA
REDIS LISTS: RETRIEVING
Using LPUSH & LTRIM

trim keeps first two strings
and removes the rest
Retrieving from list

client.lrange("messages", 0, -1, function(err, messages){
console.log(messages);
})
["Hello, no this is spider", "Oh sorry, wrong number"]
replies with all strings in list
var message = "Oh sorry, wrong number";
client.lpush("messages", message, function(err, reply){
client.ltrim("messages", 0, 1);
});
PERSISTING DATA
CONVERTING MESSAGES TO REDIS
var storeMessage = function(name, data){
messages.push({name: name, data: data});
if (messages.length > 10) {
messages.shift();
}
}
Let’s use the List data-structure
app.js
PERSISTING DATA
CONVERTING STOREMESSAGE
var storeMessage = function(name, data){
}
var redisClient = redis.createClient();
var message = JSON.stringify({name: name, data: data});
redisClient.lpush("messages", message, function(err, response) {
redisClient.ltrim("messages", 0, 10);
});
keeps newest 10 items
app.js
need to turn object into string to store in redis
PERSISTING DATA
OUTPUT FROM LIST
client.on('join', function(name) {
});
messages.forEach(function(message) {
client.emit("messages", message.name + ": " + message.data);
});
app.js
PERSISTING DATA
OUTPUT FROM LIST
client.on('join', function(name) {
});
messages.forEach(function(message) {
client.emit("messages", message.name + ": " + message.data);
});
redisClient.lrange("messages", 0, -1, function(err, messages){
messages = messages.reverse();
message = JSON.parse(message);
});
app.js
reverse so they are emitted
in correct order
parse into JSON object
PERSISTING DATA
IN ACTION


PERSISTING DATA
CURRENT CHATTER LIST
Sets are lists of unique data
client.sadd("names", "Dog");
client.sadd("names", "Spider");
client.sadd("names", "Gregg");

client.srem("names", "Spider");
client.smembers("names", function(err, names){
console.log(names);
});

["Dog", "Gregg"]
reply with all members of set
add & remove members of the names set


PERSISTING DATA
ADDING CHATTERS
client.on('join', function(name){
client.broadcast.emit("add chatter", name);
redisClient.sadd("chatters", name);
});
app.js
notify other clients a chatter has joined
add name to chatters set
index.html
server.on('add chatter', insertChatter);
var insertChatter = function(name) {
var chatter = $('

'+name+'

').data('name', name);
$('#chatters').append(chatter);
}

PERSISTING DATA
ADDING CHATTERS (CONT)
client.on('join', function(name){
client.broadcast.emit("add chatter", name);
redisClient.sadd("chatters", name);
});
app.js
notify other clients a chatter has joined
add name to chatters set
emit all the currently logged in chatters
to the newly connected client
redisClient.smembers('names', function(err, names) {
names.forEach(function(name){
client.emit('add chatter', name);
});
});


PERSISTING DATA
REMOVING CHATTERS
client.on('disconnect', function(name){
app.js
index.html
client.get('nickname', function(err, name){
client.broadcast.emit("remove chatter", name);
redisClient.srem("chatters", name);
});
});
remove chatter when they disconnect from server
server.on('remove chatter', removeChatter);
var removeChatter = function(name) {
$('#chatters li[data-name=' + name + ']').remove();
}
PERSISTING DATA
WELCOME TO CHATTR