Javascript is everywhere !

Case study : geektic* written entirely in Javascript and powered by

• Angular.js for the frontend
• Node.js for the backend
• MongoDB for the NoSQL database

* geektic was originaly developed by the code-story team for DevoxxFR 2013 (see https://github.com/CodeStory/code-story-geektic)

4 main parts :

• MongoDB : "NoSQL loves JSON"
• Node.js : "Server-side JS"
• Angular.js : "Superheroic Javascript MVW Framework"
• grunt, make and co. : "JS tooling is good !"

Exercises :

• A Git repository with some code and todos
• One tag per exercise
• Reset your workspace with the following command ("n" is the exercise's number):

git checkout -f exercise-n

• A command line tool!
• A Make tool
  • make -v must work

make -v
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

For windows users

• Git for windows comes with a command line tool (see next slide)
• For the make tool, you can install mingw32-make

• Git : see http://git-scm.com/
  • git --version must work

git --version
git version 1.7.9.5

• Node.js : see http://nodejs.org/
  • node -v and npm -v must work

node -v
v0.10.7

npm -v
1.2.21

• MongoDB : see http://docs.mongodb.org/manual/installation/
  • mongo --version must work

mongo --version
MongoDB shell version: 2.2.4

• Installation
  • Unzip the bundle
  • add the bin directory to your PATH

MongoDB is a document-oriented NoSQL database.

This kind of JSON documents will be inserted into the Mongo database :

{
    "firstname" : "Prunier",
    "lastname" : "Sébastien",
    "email" : "me@my-domain.com",
    "city" : "Nantes",
    "likes" : ["Javascript", "Breizhcamp"],
    "hates" : ["Rain"]
}

Use the --dbpath argument to set the mongo data directory

mongod --dbpath=/home/sebprunier/data/mongo/breizhcamp-js/

Check the logs in the console

...
Tue Jun  4 16:47:17 [websvr] admin web console waiting for connections on port 28017
Tue Jun  4 16:47:17 [initandlisten] waiting for connections on port 27017

Connect to the geeksDB database

mongo geeksDB

Great ! Here is the mongo shell !

MongoDB shell version: 2.2.4
connecting to: geeksDB
>

Insert geeks into the geeks collection :

db.geeks.insert({"firstname": "Prunier", "lastname": "Sébastien", 
        "email": "seb@domain.com", "city": "Nantes", 
        "likes" : ["java","javascript","breizhcamp"], "hates": ["fish"]})

db.geeks.insert({"firstname": "Seignard", "lastname": "Xavier", 
        "email": "xav@domain.com", "city": "Nantes", 
        "likes" : ["javascript","arduino","node.js"], "hates": ["scala", "idea"]})

db.geeks.insert({"firstname": "your first name", "lastname": "your last name", 
        "email": "your email", "city": "your city", 
        "likes" : ["things you like"], "hates": ["things you hate"]})

Execute this query to find a geek :

db.geeks.findOne()

Results should look like :

{
    "_id" : ObjectId("51ae04733579e9826523e0fb"),
    "firstname" : "Prunier",
    "lastname" : "Sébastien",
    "email" : "seb@domain.com",
    "city" : "Nantes",
    "likes" : ["java", "javascript", "breizhcamp"],
    "hates" : ["fish"]
}

Geeks that love javascript (ignoring case)

db.geeks.find( { "likes" : /^javascript$/i } )

First 3 geeks that love javascript

db.geeks.find( { "likes" : /^javascript$/i } ).limit(3)

3 geeks that love javascript, skiping the first one

db.geeks.find( { "likes" : /^javascript$/i } ).limit(3).skip(1)

• Checkout the workspace for exercise 1

git checkout -f exercise-1

• Play with the Mongo shell !
  • Insert geeks
  • Execute some find queries
  • Try to write a find query that uses a regular expression

• Javascript on the server side !
• Single threaded
• Event-driven model
• Asynchronous I/O
• Google's V8 Javascript engine
• NPM : Node Packaged Modules

Create a file named hello.js

var http = require('http');

http.createServer(function (req, res) {
    res.writeHead(200, {'Content-Type': 'text/plain'});
    res.end('Hello World\n');
}).listen(8888, 'localhost');

console.log('Server running at http://localhost:8888/');

Run the script

node hello.js

Test with your favorite browser (http://localhost:8888/) !

Install the native driver with NPM (Node Packaged Modules)

• npm install mongodb --save
• the --save option adds the package in the dependencies section of the package.json file

{
  "name": "breizhcamp-js_backend",
  "dependencies": {
    "mongodb": "~1.3.2",
  },
  "engines": {
    "node": "0.10.x",
    "npm": "1.2.x"
  }
}

var MongoClient = require('mongodb').MongoClient;

var uri = require('../conf/conf').MONGO_URL;

MongoClient.connect(uri, function(err, db) {
    // err : Error object, if an error occured
    // db : use this object to query the Mongo database !
}

Configuration

var Conf = {
    MONGO_URL : process.env.MONGO_URL || 'mongodb://localhost:27017/geeksDB'
};

module.exports = Conf;

db.collection('geeks', function(err, collection) {
    collection.remove({}, function(err, removed){
        console.log(removed + " geek(s) removed !");
    });
}

With the Mongo shell :

db.geeks.remove({});

Geeks are available in the geeks.json file

var geeks = require('./geeks.json');

db.collection('geeks', function(err, collection) {
    collection.insert(geeks, {safe : true}, function(err, result) {
        console.log(result.length + " geek(s) inserted !");
    });
}

safe : the callback is executed after the geeks are saved to the database

• Checkout the workspace for exercise 2 :

git checkout -f exercise-2

• Write a script to populate the geeksDB database, from the geeks.json file.
• Check the script execution with the Mongo shell !

• To go further : if you dive into the callback hell and do not like it, try async !

async = require('async');
async.series(
    [
        // 1- remove geeks
        function(callback) {...};
        // 2- insert geeks
        function(callback) {...};
    ],
    function(err, results) {...}
);

POST /geek

• Creates a new geek
• Returns the 201 - Created status code

GET /geek/likes/:like?

• Find geeks by affinity
  • like is an optional path parameter. If not set, it returns all the geeks.
• Optional query parameters :
  • limit : number of geeks to return (default 12)
  • skip : offset to manage pagination (default 0)

var express = require('express');
var app = express();

// configure routes
app.get('/hello', function(req, res){
  res.send('Hello World');
});

// start server
app.listen(3000);
console.log('Listening on port 3000');

Run the script and make a test with your browser (http://localhost:3000/hello)

One module for the routes management

var GeeksRoutes = function(geeksRepo) {

    var _create = function(req, res) {
        geeksRepo.insert(req.body, function() {
            res.status(201).send();
        });
    };

    return {
        create : _create,
    };

};
module.exports = GeeksRoutes;

One module for the repository management

var GeeksRepository = function(dbUrl, collectionName) {
    var MongoClient = require('mongodb'), db, coll;
    // some stuff here to connect to the database and retrieve the coll object.

    var _insert = function(geek, callback) {
        coll.insert(geek, function(err, item) {
            callback(err, item);
        });
    };

    return {
        insert : _insert,
    };
};
module.exports = GeeksRepository;

Put all together !

var express = require('express'),
    conf = require('./conf/conf'),
    app = express(),
    GeeksRepository = require('./core/geeksRepository'),
    GeeksRoutes = require('./routes/geeksRoutes');

// configure geeks repository
var geeksRepository = new GeeksRepository(conf.MONGO_URL, 'geeks');
geeksRepository.connect();

// configure routes
var routes = new GeeksRoutes(geeksRepository);
app.post('/geek', routes.create);

Tests for the routes, using some custom mock objects.

var assert = require('assert'),
    GeeksRoutes = require('../../src/routes/geeksRoutes'),
    geeksRepository = require('../mocks/geeksRepository.mock'),
    Response = require('../mocks/response.mock'),
    routes = new GeeksRoutes(geeksRepository);

describe('GeeksRoutes', function() {
    describe('#create()', function() {
        it('should create geek', function() {
            var response = new Response();
            var req = { body : { "NOM" : "test-geek" } };
            routes.create(req, response);
            assert.equal(response.getStatus(), 201);
        });
    });
});

• Mocha : Test Driven Development for Javascript

mocha -R spec `find test/ -name "*.test.js"`

• JSHint : static code analysis for Javascript

jshint src test --show-non-errors

• YuiDocs : generates code documentation

yuidoc src -o reports/docs

• Istanbul : code coverage

• Checkout the workspace for exercise 3 :

git checkout -f exercise-3

• Write the code to manage the find route
• Complete the unit tests
• Execute the test with the Makefile
  • make test
• Run the app and test the REST API
  • with your browser
  • with a tool to make http requests (curl for example)

Node.js overview :

• http://nodejs.org/api/ RTFM FTW!
• http://www.nodebeginner.org/
• http://visionmedia.github.io/masteringnode/book.html by TJ Holowaychuk (express & mocha creator)

Node.js good practices :

• http://nodeguide.com/style.html by Felix Geisendörfer (node.js core developer)
• http://caolanmcmahon.com/posts/nodejs_style_and_structure/ by Caolan McMahon (async creator)

• Superheroic Javascript Framework !
• Angular's Philosophy :
  • Declarative programming for UI (HTML)
  • Imperative programming for business logic (Javascript)
  • Two-way data binding
  • No DOM manipulation
  • HTML extension (directives)
  • Focus on application testing

A solution : Angular Seed !

This is a skeleton for a typical AngularJS web app

• HTML template and views
• Controller
• Examples of code for directives, filters and services
• Unit tests and End-to-end tests

Clone the Github repository and start hacking ...

https://github.com/angular/angular-seed

app/index.html

<html lang="en" ng-app>
<head>
    <meta charset="utf-8">
    <title>Geektic</title>
</head>
<body ng-controller="GeeksListCtrl">
    <div ng-repeat="geek in geeks">
        <h4>{{geek.lastname}} {{geek.firstname}}</h4>
        <p>From {{geek.city}}</p>
    </div>
    <script src="lib/angular/angular.js"></script>
    <script src="js/controllers.js"></script>
</body>
</html>

app/js/controllers.js

function GeeksListCtrl($scope) {
    $scope.geeks = [
        {...},
        {...}
    ];
}

Remember ...

<body ng-controller="GeeksListCtrl">

and

<div ng-repeat="geek in geeks">

test/unit/controllersSpec.js

describe('breizhcamp-js backend controllers', function() {
  describe('GeeksListCtrl', function(){
    var scope, ctrl;

    beforeEach(function() {
      scope = {}, ctrl = new GeeksListCtrl(scope);
    });

    it('should create "geeks" model with 3 geeks', function() {
      expect(scope.geeks.length).toBe(3);
    });
  });
});

Take a look at scripts/test.sh and run it !

test/e2e/scenario.js

describe('breizhcamp-js backend app', function() {
  describe('Geeks list view', function() {

    beforeEach(function() {
      browser().navigateTo('../../app/index.html');
    });


    it('should filter the geeks list as user types into the search box', function() {
      expect(repeater('.geeks div').count()).toBe(3);
    });
  });
});

Take a look at scripts/e2e-test.sh and run it !

• Checkout the workspace for exercise 4 :

git checkout -f exercise-4

• Complete the model with some "static geeks"
• Complete the index.html view to rendre the geeks
• Play with the filtering
  • add an input text to filter the data
  • use the filter in the ng-repeat directive

<input ng-model="query" />
...
<div ng-repeat="geek in geeks | filter:query" >

With the angular's $http service, it's easy to make HTTP requests to the backend.

function GeeksListCtrl($scope, $http) {
    $http.get('geek/likes').success(function(data){
        $scope.geeks = data;
    });
}

• Checkout the workspace for exercise 5 :

git checkout -f exercise-5

• Use the $http service to request the backend
• Use the ng-change directive to update the model when the search query changes

To go further ...

• Try to use the $resource service

var Geeks = $resource('/geek/likes/:like?');
$scope.geeks = Geeks.get({like : $scope.query});

• A POM (Plain Old Makefile), not a pom.xml!
• Get the things done
• Define targets for each task, e.g. clean, test, linting, etc.

clean:
    rm -rf reports

• Define variables that can be reused in tasks

MOCHA="node_modules/.bin/mocha"
TESTS=$(shell find test/ -name "*.test.js")

test:
    $(MOCHA) -R spec $(TESTS)

• Run a target with make target

• Jenkins ready!

• The tooling we use, can produce various reports (unit testing, checkstyle, coverage, ...)

• Define a continuous integration target that call other targets

ci: clean xunit checkstyle coverage sonar

• Various outputs when running the tests : spec, xunit, nyan cat, etc. (see: http://visionmedia.github.io/mocha/#reporters)

• You can use buit-in reporters or extend Mocha by creating your own reporter.

• Install the xunit-file reporter to produce xunit compliant file

npm install xunit-file --save-dev

• TODO : Create a Makefile target that produce the xunit file
  • see the docs here : https://github.com/peerigon/xunit-file

• Pretty straightforward, but note the double dash to distinguish istanbul args from the mocha ones and the use of _mocha internal executable (see istanbul/issues/44)

istanbul cover _mocha -- -R spec test/**/*.test.js

• TODO : Create a Makefile target that only produce a lcov file

• lcov format is directly usable by Sonar

• To display coverage results in Jenkins, use lcov-to-cobertura + jenkins coverage plugin

• Bower is a repository of packaged components (JS, CSS, or whatever)
• Defined in bower.json.

• You can depend on various format of assets

  • Registered asset within bower : bower install jquery
  • Files : bower install http://domain.com/myFile.js
  • Archives : bower install http://domain.com/myArchive.zip
  • Github repo : bower install repoOwner/repo
  • Github tag/commit bower install jquery#1.9.1

• TODO : create the bower.json file with bower init and manage angular libs with bower.

• Use bower search and bower install XXX --save

• Grunt is a JS task runner.
• Install grunt-cli globally, and grunt as a dev dependency in your project.

npm install grunt-cli -g;npm install grunt --save-dev

• Create a Gruntfile.js

module.exports = function(grunt) {
  // project configuration
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json')
    // the tasks definition goes here
  });
  grunt.registerTask('default', []);
};

• Run grunt (will do nothing but without errors :))

• Plugin based
• Official ones start with grunt-contrib, see http://gruntjs.com/plugins
• Add the clean plugin

npm install grunt-contrib-clean --save-dev

• And add the following in Gruntfile.js to make the task available

grunt.loadNpmTasks('grunt-contrib-clean');

• You can now define the clean task (JSON object or array) and plug this task in your build lifecycle.

module.exports = function(grunt) {
  // project configuration
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json')
    // the tasks definition goes here
    clean: ['myDistDir']
  });
  grunt.registerTask('default', [clean]);
  grunt.loadNpmTasks('grunt-contrib-clean');
};

• The command grunt is a shorthand for grunt default. You can also call single tasks, e.g. grunt clean

• You can define sub and super tasks inside your task to be more specific

module.exports = function(grunt) {
  // project configuration
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json')
    // the tasks definition goes here
    clean: {
      build: ['myDistDir'],
      postbuild: ['myDistDir/tmp']
    }
  });
  grunt.registerTask('default', [clean:build]);
  grunt.registerTask('buildCleanUp', [clean:postbuild]);
  grunt.registerTask('mySuperTask', [default, buidCleanUp]);

  grunt.loadNpmTasks('grunt-contrib-clean');
};

It mainly consists on the following tasks:

• lint your code
• run tests
• concatenate your JS and CSS files
• minify your code
• copy all the assest and created files in a dist folder

You can do much more to fit exactly your needs.

We are going to change of branch, save and commit what you want to keep.

git checkout master

And open client/Gruntfile.js.

TODO1 : try to run some of the defined tasks

TODO2 : adapt the watch task to have an always up-to-date build dir.

See livereload or the livereload script definition in client/src/index.html

• http://gruntjs.com/getting-started RTFM FTW!
• Amazing API : http://gruntjs.com/api/grunt
• Super easy to create your own tasks : http://gruntjs.com/api/grunt.task#grunt.task.registertask
• A ton of plugins!

We are able to define super tasks with make and grunt so you just need to create a ci task for each.

For QA you can generate reports from the tools involved in the build, or use Sonar.

Sonar digests xunit files for test results and lcov files for coverage, and we shown how to generate these ones. Just install the Javascript plugin

• Use nginx (or similar) to serve static content and as a reverse proxy for your node.js server.
• Use git post commit hooks (or a secure copy) to update your production code.

• Use a service manager to run your app

  • init, upstart, systemd

• Icing on the cake : try Socket based activation! Run your app only when people are using it. (see systemd.socket as potential solution)