Category Archives: Automation

Alternate way Automated node.js Deployments to Bluemix via Jenkins

After having used the CloudFoundry plugin for Jenkins for a bit we found that the BM API was periodically erroring out and returning such things as:

400 Bad Request
and
Java.net.SocketTimeoutException: Read timed out
and
502 Bad Gateway

So I installed the cf tools on the jenkins server and was many times able to get successful build pushed to bluemix by running an Excecute Shell in the Build Step with the following commands:

cd public;
npm install;
gulp build;
cd ../;
sudo cf api https://api.ng.bluemix.net;
sudo cf login -u [email protected] -o organization.com -s staging -p password;
sudo cf push MyNodeApp-staging;

If you are binding to services, like mongodb, the trick is to have a manifest.yml file in the root of the project (should be in .gitignore if you pushing multiple branches) In this case the manifest looks like:

applications:
- services:
  - mongodb-H
  disk_quota: 1024M
  host: MyNodeApp-staging
  name: MyNodeApp-staging
  path: .
  domain: mybluemix.net
  instances: 1
  memory: 256M
env:
  NODE_ENV: staging

Where the service name to bind to is specified and the environment variable NODE_ENV is passed through for the application to handle environmental case switching.

Automated node.js Deployments to Bluemix via Jenkins

One of our new clients is a partner of IBM and we are building a node.js-based application for them. Through their IBM partnership they were encouraged to leverage IBM’s Bluemix Cloud Foundry-based PaaS for development and deployment. We have plenty of experience with virtualized environments and are always interested in exploring new technologies and platforms. However, with the client’s best interest and budget taken to heart, we decided to try and use as much of our existing workflow and process as possible rather than spending a ton of hours getting ramped on the bluemix DevOps platform.

So we have our codebase checked into bitbucket just like every other project and we are running jenkins hooks to our CI server to trigger builds on check in. There a wonderful Jenkins plugin for Cloud Foundry code pushes, which works perfectly for pushing to Bluemix – https://wiki.jenkins-ci.org/display/JENKINS/Cloud+Foundry+Plugin

Once the plugin is installed via Jenkins plugin manager, it should appear as a “Post Build Action” in the project configuration screen. (If you install the plugin and that option does not show up, Jenkins is likely pointed at a java version prior to 1.7 – you can confirm this by looking in the jenkins logfile for an error about “Unsupported major.minor version 51.0″)

Once you get the option, you simply add your bluemix credentials via the credential manager, then set up the Cloud Foundry post build action with the correct values for the parameters:

Capture

And the web app gets pushed to bluemix, which takes care of all the vendor dependencies and deploys the app!

Checking out Revision d846a37fc9b234f37713005dc88a7ba78855f3d6 (origin/staging)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f d846a37fc9b234f37713005dc88a7ba78855f3d6
 > git rev-list d846a37fc9b234f37713005dc88a7ba78855f3d6 # timeout=10
Cloud Foundry Plugin:
Pushing SDStaging app to https://api.ng.bluemix.net
App already exists, resetting.
App deleted.
Creating new app.
Pushing app bits.
Starting application.
       Installing IBM SDK for Node.js (0.12.2) from cache
       Using default npm version: 2.7.4
-----> Checking and configuring service extensions before installing dependencies
-----> Building dependencies
       Installing node modules
       > (node-gyp rebuild 2> builderror.log) || (exit 0)
       > [email protected] install /tmp/staged/app/node_modules/mongoose/node_modules/mongodb/node_modules/mongodb-core/node_modules/kerberos
       make: Entering directory `/tmp/staged/app/node_modules/mongoose/node_modules/mongodb/node_modules/mongodb-core/node_modules/kerberos/build'
         CXX(target) Release/obj.target/kerberos/lib/kerberos.o
         CXX(target) Release/obj.target/kerberos/lib/worker.o
         CC(target) Release/obj.target/kerberos/lib/kerberosgss.o
         CC(target) Release/obj.target/kerberos/lib/base64.o
         CXX(target) Release/obj.target/kerberos/lib/kerberos_context.o
         SOLINK_MODULE(target) Release/obj.target/kerberos.node
         SOLINK_MODULE(target) Release/obj.target/kerberos.node: Finished
         COPY Release/kerberos.node
       make: Leaving directory `/tmp/staged/app/node_modules/mongoose/node_modules/mongodb/node_modules/mongodb-core/node_modules/kerberos/build'
       > (node-pre-gyp install --fallback-to-build) || (node-gyp rebuild 2> builderror.log) || (exit 0)
       > [email protected] install /tmp/staged/app/node_modules/mongoose/node_modules/bson/node_modules/bson-ext
       make: Entering directory `/tmp/staged/app/node_modules/mongoose/node_modules/bson/node_modules/bson-ext/build'
         CXX(target) Release/obj.target/bson/ext/bson.o
         SOLINK_MODULE(target) Release/obj.target/bson.node
         SOLINK_MODULE(target) Release/obj.target/bson.node: Finished
         COPY Release/bson.node
       make: Leaving directory `/tmp/staged/app/node_modules/mongoose/node_modules/bson/node_modules/bson-ext/build'
       [email protected] node_modules/vhost
       [email protected] node_modules/debug
       └── [email protected]
       [email protected] node_modules/cookie-parser
       └── [email protected]
       ├── [email protected]
       └── [email protected]
       ├── [email protected]
       ├── [email protected]
       └── [email protected]
       [email protected] node_modules/serve-favicon
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       [email protected] node_modules/passport
       [email protected] node_modules/body-parser
       └── [email protected] ([email protected])
       [email protected] node_modules/morgan
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected] ([email protected])
       ├── [email protected] ([email protected])
       ├── [email protected]1 ([email protected], [email protected])
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       [email protected] node_modules/express
       └── [email protected] ([email protected], [email protected])
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected] ([email protected], [email protected])
       ├── [email protected]
       ├── [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected])
       ├── [email protected]
       ├── [email protected] ([email protected])
       [email protected] node_modules/hbs
       ├── [email protected] ([email protected])
       [email protected] node_modules/mongoose
       ├── [email protected] ([email protected], [email protected])
       ├── [email protected]
       ├── [email protected]
       ├── [email protected] ([email protected])
       └── [email protected] ([email protected], [email protected], [email protected])
       ├── [email protected]
       └── [email protected] ([email protected], [email protected])
       ├── [email protected]
       ├── [email protected]
       └── [email protected] ([email protected])
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected] ([email protected])
       ├── [email protected]
       ├── [email protected] ([email protected], [email protected])
-----> Checking startup method
       No Procfile; Adding 'web: npm start' to new Procfile
-----> Checking and configuring service extensions after installing dependencies
-----> Installing App Management
       Creating runtime environment
-----> Finalizing build
-----> Building runtime environment
       Caching results for future builds
       Cleaning previous cache
-----> Build succeeded!
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       ├── [email protected]
       [email protected] /tmp/staged/app
       ├── [email protected]
       └── [email protected]
       ├── [email protected]
       ├── [email protected]
       WARNING: Node version not specified in package.json

DevOps Automation – Ansible+Semaphore is Indispensable!

UPDATE
Semaphore 2.x is in current release and much of the information regarding the install process below is no longer relevant as the package has been significantly refactored. The developers have made it much easier to install: See the install directions here: https://github.com/ansible-semaphore/semaphore/wiki/Installation

I deviated a bit, but got it up and running in less than 5 minutes. Here’s how:

As root:

#cd /usr/bin
#wget https://github.com/ansible-semaphore/semaphore/releases/download/v2.0.2/semaphore_linux_amd64
#mv semaphore_linux_amd64 semaphore
#chmod a+x semaphore
#semaphore -setup

semaphorescreen1
semaphorescreen2
semaphorescreen3

Note that I already had mysqld running on the server.

As Spire Digital delves deeper in DevOps, I have had the opportunity to test drive a number of automation technologies – Puppet and Chef, of course.  But the open-source, python-based ansible project is far and away my preferred technology for automating server configuration and initial web app deployments and configuration.  It even handles could-based instance provisioning!  One of it’s best features is that it needs no client agent running on the target server – everything is done through ssh using key-based authentication.  We run redhat-flavored VMs (amazon-linux-ami) so installing ansible goes like this:

#sudo yum install epel-release
#sudo yum-config-manager --enable epel
#sudo yum install ansible

And then you have all of ansible’s cli tools available to issue ad-hoc commands via ansible or run playbooks using ansible-playbook.  It gets even better with a GUI.  And while I think that the ansible tower GUI is reasonably priced (especially when compared to puppet’s GUI), I found the semaphore project on github to be lean and very useful – it’s not yet well documented, so you do have to deal with some guesswork or hope the developers will answer questions on the github board for the project, but once you get everything configured and understand how it works, it’s great!

The semaphore application is a node.js app that normally runs under vagrant or docker – I skipped the vagrant/docker, and just set it up as a node.js app directly on a fresh dedicated server.  Two gotchas:

1. need to npm install some modules for dependencies:
async body-parser connect-redis express jade logtrail morgan passport.socketio socket.io bcrypt bugsnag cookie-parser express-session less-middleware mongoose passport serve-static validator

2. need to copy lib/credentials.default.json to lib/credentials.json

Then spin it up running node bin/semaphore.js  -  it will bind to port 80 by default, but you can change that in lib/config.js by changing the exports.port variable.  Point a browser at your install and log in!

What does semaphore do?  So basically it runs playbooks for you.  And here’s how:  Put your playbooks in a git repository (we use bitbucket)  There are only 2 top-level navigation items in semaphore:  Playbooks (which point to the git project)  and Identities (which are private/public ssh keypairs)  You enter keys for the git repository as an identity, then you enter the URL to your repo containing your playbooks and associate it with the right identity.

playbook

Semaphore does not use ansible’s default hosts file, so you’ll need to enter hosts and host groups directly into semaphore.

Now you need to enter a Job.  A job is a playbook in your repo – for example if you want to run a playbook from your repository that lives in a subdirectory of the project called “project-deploy” you would enter project-deploy/software-deploy.yaml  as the “Play” (“Name” can be whatever.)

 

jobjobs

When you click on run for that job, semaphore pulls down the repo into a temp folder and runs ansible-playbook on the .yaml file that you specified – you can monitor the output of the job by going to tasks > see outputtasksCapture

*Protip – semaphore will use the same ssh keypair for fetching the git repo as it will for connecting to the remote host, so you need to be sure that the public key for your git account is also in the authorized_keys file on your target host. You may also need to add an ssh config file to /root/.ssh/config that looks like this:

Host *
  StrictHostKeyChecking no
  CheckHostIp no
  PasswordAuthentication no
  PreferredAuthentications publickey

*Protip2 – semaphore downloads git repos into /root/ as configured in vagrant or docker that works fine, but if you’re working on an install outside of a VM, you need to run semaphore as root.

Here’s an example playbook that ensures a server has everything we need to run our app:

---
- hosts: testservers
# set the php version that needs to run on the server here
# and ansible will pull in the relevant vars file for that version
# it does not matter if php is currently on the image as this playbook
# completely uninstalls any version and replaces it with the version
# specified here.
# [email protected] - 05/13/15
vars:
phpversion: 55
vars_files:
- software-vars-php{{ phpversion }}.yaml
gather_facts: true
remote_user: ec2-user
become: yes

tasks:
- name: Install EPEL repo.
yum: name={{ repo_package }} state=latest

- name: enable EPEL.
shell: yum-config-manager --enable {{ repo_name }}

- name: clean up any existing php.
yum: name=php* state=absent

- name: clean up any existing php54.
yum: name=php54* state=absent

- name: clean up any existing php55.
yum: name=php55* state=absent

- name: clean up any existing php56.
yum: name=php56* state=absent

- name: Next - update all core packages
yum: name=* state=latest

- name: Then, install required system packages.
yum: name={{ item }} state=latest
with_items: "{{ system_packages }}"

- name: Now, install required npm packages.
npm: name={{ item }} global=yes
with_items: "{{ npm_packages }}"

- name: Install apache security configuration file.
copy: src=conf/x-secure.conf dest=/etc/httpd/conf.d/x-secure.conf

- name: And, restart httpd services.
service: name=httpd state=restarted

- name: Almost done - now enable httpd service at boot.
shell: chkconfig httpd on

- name: Finally, enable mysqld service at boot.
shell: chkconfig mysqld on

And here is the associated variables file:

---
repo_package: epel-release
repo_name: epel
system_packages:
- git
- httpd24
- httpd24-tools
- httpd24-devel
- mod24_ssl
- php55
- php55-bcmath
- php55-cli
- php55-common
- php55-dba
- php55-devel
- php55-gd
- php55-imap
- php55-intl
- php55-ldap
- php55-mbstring
- php55-mcrypt
- php55-mysqlnd
- php55-odbc
- php55-pdo
- php55-pecl-jsonc
- php55-pecl-jsonc-devel
- php55-pecl-memcached
- php55-pecl-oauth
- php55-pecl-redis
- php55-pecl-ssh2
- php55-pgsql
- php55-pspell
- php55-soap
- php55-tidy
- php55-xml
- php55-xmlrpc
- mysql55-server
- redis
- npm
- nodejs
- mongodb
npm_packages:
- bower
- npmlog
- node-uuid

 

Git yourself some semaphore here: https://github.com/ansible-semaphore/semaphore