Comminty Build Framework

Focused on a multi-project / multi-environment scenario, CBF is the way to setup and deploy Pentaho-based applications.

CBF is extremely useful when working in several projects at the same time, allowing you to switch among several environments (production, development, etc.), even when customizing the Pentaho server (using different databases, changing the look and feel or applying different security types).

Credits: Webdetails Team, Lead - Pedro Alves;

Documentation

Using Pentaho's distribution packages may be a very fast way to get a quick up and running server. However, one can easily get lost to which changes were made to build the project.

This will be specially painful when we want to upgrade. Also, for collaboration between distinct team members, just sharing the solution's folder isn't enough, as lots of settings get lost.

CBF targets this and other requirements that come from a real world need. By using a predefined directory structure and building Pentaho from source, we have full control over our installation.

Main Features

  • Ability to switch from totally different projects on the fly;
  • Supports multiple environments (eg: development, production, worker1, worker2...);
  • No changes to the original files that could get overwritten in a upgrade;
  • Supports multiple platform versions;
  • Simplifies version upgrades;
  • Debug-friendly;
  • VCS (Version Control System)-friendly;
  • Supports deploy to local/remote machines.
  • Supports all kind of different customization in different projects, from using different databases to different security types;
  • Supports patches to the source code for fine-grain customization;
  • OS independent.

CBF Requirements

  • A shell (if on Windows, Cygwin or equivalent is required);
  • Clean directory to host your CBF installation;
  • Jdk 1.6.;
  • Ant 1.7.;
  • SVN client.
  • Tomcat 6.0.x (for pentaho 3.7 or before, tomcat 5.5 is required);
  • RSync / SSH to do remote deploy.

We need to create a directory where our CBF structure will live. This directory, which I'll call cbf, will have:

  • The CBF script;
  • Pentaho source code;
  • Your project directory.

The CBF build script is a tweaked build.xml file (download it form the download section). Specific project directories have the format project-<something> . In this example we'll use project-client . So the structure looks like:

cbf
 ├── pentaho-1.6
 ├── pentaho-1.7
 ├── pentaho-2.0
 ├── pentaho-3.0
 ├── pentaho-3.5
 ├── pentaho-3.5.2
 ├── pentaho-3.6
 ├── pentaho-3.7
 ├── pentaho-3.8
 ├── project-bare17
 ├── project-bare20
 ├── project-bare30
 ├── project-bare30-training
 ├── project-bare35
 ├── project-bare352
 ├── project-bare36
 ├── project-bare37
 ├── project-bare38
 └── build.xml

Pentaho Source

Pentaho source can be downloaded using a SVN client. I use command line tool, so it would be something like this:

svn co svn://source.pentaho.org/svnroot/bi-platform-v2/tags/3.8.0-stable pentaho-3.8

Alternatively, http instead of svn can be used if you're behind a firewall. You can browse the SVN tree to get other versions, though sometimes it isn't easy to find the correct one.

Project Client Directory

This is where we'll put our files. If you're starting a new project, create this from scratch.

You can also start from the Quickstart section for a pre-built structure. You'll end up with something like this:

project-client
 ├── config
 │ ├── build-nelson.properties
 │ ├── build-neraka.properties
 │ ├── build-paula.properties
 │ ├── build-pdpi.properties
 │ ├── build-pedro.properties
 │ ├── build-tgf.properties
 │ └── build.properties
 ├── patches
 │ ├── project-client
 │ ├── target-build
 │ └── target-dist
 └── solution
  ├── MySolution
  ├── admin
  ├── bi-developers
  ├── cdf
  ├── index.properties
  ├── index.xml
  ├── steel-wheels
  └── system

The directories in italic are generated during the build process. There are 3 main blocks:

Config

Here lives the different configuration files that will control the build process. When executing CBF, we'll pass an option that defines which environment will be used, like -Denv=pedro . In all cases, the build.properties will be read first for common properties and then the specific file will override this values. In this example, the build.properties file will be read followed by build-pedro.properties .

This example was taken from a Webdetails project, so you'll see several files for each person that has worked with it. We could also have separate environment configuration files for deploying to server.

Patches

The patches directory will be used for three purposes:

  • Making changes to the solution/system configuration;
  • Patching the server dir (such as adding jars or changing log4j.xml);
  • Patching the source (advanced - for changing java files).

Inside the patches dir you'll often see 3 main directories:

  • project-client;
  • target-build;
  • target-dist.

Solution

This is our proper solution implementation. Use as many as needed, but take care to avoid changing the admin , bi-developers , steel-wheels and system directories , as they are wiped and copied again during the build process. For changes in these specific directories, use the patches directory.

The directory structure has to obey the CBF structure. project-client has to match your project name. This is an important detail for the patches to be applied correctly. target-build and target-dist are two CBF specific directories. The first one holds the Pentaho source to be compiled and the second is where the server will be installed into.

Here's a real world example that should be easy enough to understand and extend:

patches
├── project-client
│   └── solution
│       └── system
│           ├── applicationContext-security-ldap.properties
│           ├── applicationContext-spring-security-hibernate.properties
│           ├── applicationContext-spring-security-ldap.xml
│           ├── applicationContext-spring-security.xml
│           ├── hibernate
│           │   ├── hibernate-settings.xml
│           │   ├── hsql.hibernate.cfg.xml
│           │   └── mysql5.hibernate.cfg.xml
│           ├── mondrian
│           │   └── mondrian.properties
│           ├── olap
│           │   └── datasources.xml
│           ├── pentaho-spring-beans.xml
│           ├── pentaho.xml
│           ├── pentahoObjects.spring.xml
│           ├── publisher_config.xml
│           ├── sessionStartupActions.xml
│           └── smtp-email
│               ├── email_config.xml
│               └── email_config_gmail.xml
└── target-dist
    └── server
        ├── conf
        │   └── server.xml
        ├── lib
        │   ├── LucidDbClient.jar
        │   ├── c3p0-0.9.1.2.jar
        │   ├── commons-dbcp-1.2.1.jar
        │   └── commons-pool-1.2.jar
        └── webapps
            ├── ROOT
            │   └── favicon.ico
            └── pentaho
                ├── META-INF
                │   └── context.xml
                ├── WEB-INF
                │   ├── classes
                │   │   └── log4j.xml
                │   └── web.xml
                ├── index.jsp
                └── jsp
                    └── PUCLogin.jsp

By respecting this structure we'll be able to keep our changes to the pentaho installation in a single place. This can then be placed in a VCS and shared with the team.

Configuration files play a key role in CBF. It's where all the information needed for runtime will be specified. When we run CBF script, we specify not only which project we're running: (-Dproject=client) , but also which configuration (from now on we'll use the word environment) we want to use (-Denv=pedro) . Here's a sample of calling CBF: ant -Dproject = client -Denv=pedro dist-clean all .

In this case, CBF will look for a configuration file under project-client/config/build-pedro.properties . That file contains Pedro's specific configuration and inherits all the properties defined in build.properties

Build.Properties

######################################
## GENERIC PROPERTIES
######################################
javac.path = NOT SET
#jre.dir = NOT SET
#project revision. May need to change according to project
project.revision=3.8.0-stable
dependency.pentaho-reporting.revision=3.8.0-stable
# Solution path. Required for compile time
solution.path = NOT SET
# Solution path required for runtime. Defaults to the above value but can be different if we're
# deploying to another place. Should be used in WEB-INF/web.xml in the solution-path parameter
solution.deploy.path = NOT SET
# Copy pentaho samples (bi-developers and steelwheels) to target-dist at runtime?
copy.pentaho.samples = true
# Tomcat 6 path:
tomcat.path = NOT SET
# Uncomment to use latest CDF from trunk
dependency.pentaho-cdf-plugin.revision=TRUNK-SNAPSHOT
# Java options for the run command
java.opts = -Dfile.encoding=utf-8 -Xmx512m -XX:MaxPermSize=512m -Xrunjdwp:transport=dt_socket,address=8765,server=y,suspend=n
######################################
## PROJECT DIRECTORIES - The defaults usually work fine
######################################
pentaho.dir = pentaho-3.8/
pentaho.build.dir = target-build/
######################################
## DEPLOY OPTIONS
######################################
deploy.mode = rsync
deploy.args = -avz --exclude '.svn/' --exclude '*.log' --exclude 'work/' --exclude 'temp_user/' --exclude 'temp/
' --exclude 'tmp/'
deploy.dest.server = user@host:path
deploy.dest.solution = user@host:path
######################################
## DYNAMIC TOKENS TO BE USED AS FILTERS
######################################
DB_LOCATION = 127.0.0.1
DB_USER = pedro
DB_PASSWORD = bar
BASE_URL = http://127.0.0.1:8080/pentaho/
######################################
## LOGS
######################################
PENTAHO.LOG.FILE = pentaho.log
MONDRIAN.LOG.FILE = mondrian.log
PENTAHO.LOG.LEVEL = INFO
MONDRIAN.LOG.LEVEL = DEBUG
MONDRIAN.SQL.LOG.LEVEL = DEBUG

This is a fairly complete example of a build file.

Some of the properties are mandatory for the build process while others are defined to be replaced as tokens (more information regarding tokens elsewhere in this document).

The values that are defined act as default values for the build.

While everything can be overridden in the environment-specific file, they usually can be left untouched. The ones marked as "NOT SET" have to be overridden for each environment.

Explanation of some important property names:

  • pentaho.dir : source code of the server we're building;
  • project.revision : version specific property;
  • dependency.pentaho-reporting.revision : This is specific to Pentaho 3.8.0, usually not needed for other versions;
  • tomcat.path : path to a unmodified version of Tomcat. Pentaho 3.8 requires version 6.0.x while older versions work against version 5.5.x.

Build.Pedro.Properties

Here's a sample of Pedro's sandbox file:

javac.path = /usr/bin/javac
tomcat.path = /Users/pedro/tex/pentaho/apache-tomcat-6.0.32/
pentaho.dir = pentaho-3.8
solution.path = /Users/pedro/tex/pentaho/project-client/solution
solution.deploy.path = /Users/pedro/tex/pentaho/project-client/solution

Patches are a good way to change the stock Pentaho install, either by adding content (like jars or plugins) or by changing configuration files through what we call tokens.

Here's a sample structure under patches:

patches
├── project-client
│   └── solution
│       └── system
│           ├── applicationContext-security-ldap.properties
│           ├── applicationContext-spring-security-hibernate.properties
│           ├── applicationContext-spring-security-ldap.xml
│           ├── applicationContext-spring-security.xml
│           ├── hibernate
│           │   ├── hibernate-settings.xml
│           │   ├── hsql.hibernate.cfg.xml
│           │   └── mysql5.hibernate.cfg.xml
│           ├── mondrian
│           │   └── mondrian.properties
│           ├── olap
│           │   └── datasources.xml
│           ├── pentaho-spring-beans.xml
│           ├── pentaho.xml
│           ├── pentahoObjects.spring.xml
│           ├── publisher_config.xml
│           ├── sessionStartupActions.xml
│           └── smtp-email
│               ├── email_config.xml
│               └── email_config_gmail.xml
└── target-dist
    ├── administration-console
    │   ├── jdbc
    │   │   ├── LucidDbClient.jar
    │   │   ├── c3p0-0.9.1.2.jar
    │   │   ├── commons-dbcp-1.2.1.jar
    │   │   └── commons-pool-1.2.jar
    │   └── resource
    │       └── config
    │           └── console.xml
    └── server
        ├── conf
        │   └── server.xml
        ├── lib
        │   ├── LucidDbClient.jar
        │   ├── c3p0-0.9.1.2.jar
        │   ├── commons-dbcp-1.2.1.jar
        │   └── commons-pool-1.2.jar
        └── webapps
            └── pentaho
                ├── META-INF
                │   └── context.xml
                └── WEB-INF
                    ├── classes
                    │   └── log4j.xml
                    └── web.xml

There are 2 different kinds of files in the above structure:

  • Adding new files: like the LucidDB jar file and other jars;
  • Changing existing files: usually to parametrize a configuration option.

There's not much to it when it comes to adding new files, they will end up in the desired location. For parameterizing configuration files, we need to use tokens.

Tokens

The principle behind tokens is very simple; when CBF finds a file in the patches directory that contains a token - a string enclosed within @ (example: @DB.HIBERNATE.DRIVER@) it will copy it to the appropriate location with the token replaced by the actual value. That way we can build our configured Pentaho installation directly from a set of property files.

Here's how a sample files looks: project-client/patches/project-client/solution/system/hibernate/hibernate-settings.xml :

< ?xml version='1.0' encoding='utf-8'? >
< settings >
  < !--
  * This setting allows the deployment to specify where to find the
  * database-specific hibernate configuration. The samples supplied
  * include the following:
  *
  * system/hibernate/hsql.hibernate.cfg.xml
  * system/hibernate/mysql5.hibernate.cfg.xml
  * system/hibernate/postgresql.hibernate.cfg.xml
  * system/hibernate/oracle10g.hibernate.cfg.xml
  *
  -->
    < config-file>system/hibernate/@DB.HIBERNATE@.hibernate.cfg.xml</config-file >
    < managed>false</managed >
< /settings >

Using this method, depending on the value of the DB.HIBERNATE property, we can easily switch from hypersonic to MySQL or others.

There's an easy way to check which targets are supported by CBF:

$ ant -Dproject=bare38 -Denv=pedro -p

Buildfile: /Users/pedro/tex/pentaho/build.xml
     [echo] --------------------------------------------------------------
     [echo] --------------------------------------------------------------
     [echo] ------       CBF - Community Build Framework
           -------
     [echo] ------                 Version: 3.7
                    -------
     [echo] ------ Author: Pedro Alves (pedro.alves@webdetails.pt) -------
     [echo] --------------------------------------------------------------
     [echo] --------------------------------------------------------------
     [echo]
     [echo]                          SETTINGS:
     [echo]
     [echo]    Project:        bare38
     [echo]    Environment:    build-pedro.properties
     [echo]    Config File:    project-bare38/config/build-pedro.properties
     [echo]    Server:        tomcat
     [echo] --------------------------------------------------------------
     [echo] --------------------------------------------------------------
Pentaho BI Platform build helper..
Main targets:
 all                 Compiles and builds the entire project
 clean               Cleans the solution
 copy-finish         Copy target files
 copy-init           Copy project files
 deploy-all          Deploys both the solution and the server to the specified destinations
 deploy-ee-all       Deploys both the solution and the server to the specified destinations
 deploy-ee-server    Deploys the entire server to the specified destination
 deploy-ee-solution  Deploys the solution to the specified destination
 deploy-repository   Deploys the solution repository to the specified destination
 deploy-server       Deploys the entire server to the specified destination
 deploy-solution     Deploys the solution to the specified destination
 deploy-war          Deploys the pentaho.war and pentaho-style.war to the specified destination
 dev-setup           Compiles and builds the entire project
 dist-clean          Deletes the target preconfigured install dir
 run                 Runs the solution
 stage-ee            Setup Enterprise Edition staging directory
Default target: all

Even though there are a lot of targets here, some of them are internal only and on a daily basis only some of them will be used. I'll divide this section in the operations mostly used.

Doing a full compile

When we want to "start from scratch", with a clean system, this is the command to use:

$ ant -Dproject=bare38 -Denv=pedro dist-clean all

Start the project after compiled

This will simply start tomcat

$ ant -Dproject=bare38 -Denv=pedro run

Full compile and start server

You can guess this one:

$ ant -Dproject=bare38 -Denv=pedro dist-clean all run

Deploy to a remote server, solutions only

$ ant -Dproject=bare38 -Denv=stagedeploy dist-clean deploy-solutions

Apply the patches only without a full compile

No need to do the dist-clean here.

$ ant -Dproject=bare38 -Denv=pedro copy-init run

Full deploy to a remote server

This will transfer both server and solutions.

$ ant -Dproject=bare38 -Denv=pedro dist-clean deploy-all

One of the best features about CBF is the ability to deploy a full configured server (or only the solution) to the production server. Usually, every developer wants to work locally on his sandbox and at a point in time transfer the work to test and production servers.

CBF makes this task easy by using rsync over SSH to make the deploy. Use this feature carefully and at your own risk.

Requirements

  • SSH access to the server (using an account with proper permissions);
  • Environment file describing the deploy properties.

Build-server.properties

Building an environment file for deploy is very similar to the one used for local sandbox but with some important differences:

  • deploy.mode - Currently, the only supported value is rsync;
  • solution.deploy.path - The solution path on the remote server. solution.path is still the solution directory in the local machine;
  • deploy.args - The arguments to be passed to rsync. Typical arguments include: -avz --exclude '.svn/' --exclude '*.log' --exclude 'work/' --exclude 'temp_user/' --exclude 'temp/' --exclude 'tmp/' --delete ;
  • deploy.dest.server - Place where to copy the server to;
  • deploy.dest.solution - Place where to copy the solution to.

Example File

javac.path = /usr/lib/java/bin/javac
 tomcat.path = /home/pedro/tex/pentaho/apache-tomcat-6.0.29/
 pentaho.dir = pentaho-3.8
 solution.path = /home/pedro/tex/pentaho/project-bare38/solution
 solution.deploy.path = /home/webdetails/pentaho/pentaho-solution
 ######################################
 ## DEPLOY OPTIONS
 ######################################
 deploy.mode = rsync
 deploy.args = -avz --exclude '.svn/' --exclude '*.log' --exclude 'work/' --exclude 'temp_user/' --exclude 'temp/'
 --exclude 'tmp/' --delete
 deploy.dest.server = webdetails@myserver:pentaho/pentaho-server/
 deploy.dest.solution = webdetails@myserver:pentaho/pentaho-solution/

One important thing to note in this example is the '--delete' option in deploy.args . If this option is in place, every file that exists in the remote location will be wiped. While very useful if you want to make sure the version you deploy is exactly the same as the one you work with locally, it can delete unwanted information from the remote server, like saved content from users.

CBF stands for Community Build Framework — but it still fit Pentaho Enterprise Edition. If you already have a CBF environment configured, there's only a few extra steps necessary to make it work with Enterprise Edition.

In fact, all you need is to add a patches-ee folder to your project-client root, and add the deployment variables to your environment file. This particular setup deploys to the local host via rsync, and points deploys in the localhost:

deploy.mode = rsync
deploy.args = -avz --exclude '.svn/' --exclude '*.log' --exclude 'work/' --exclude 'Catalina/' --exclude
'temp_user/' --exclude 'temp/' --exclude 'tmp/'
deploy.dest.server = 127.0.0.1:/Volumes/Projects/biserver-ee
deploy.dest.solution = 127.0.0.1:/Volumes/Projects/biserver-ee/pentaho-solutions

Since these options are not required to run a local instance of the CE server, and because it's predictable that there will be differences between the local and remote environments (including database setups, JAVA location, etc), we recommend that, instead of adding all these settings to your environment file, you add a new environment file for your EE deployments.

For example, if your usual environment file is build-pedro.properties , consider build-pedro-ee.properties .

Patching

Patching for the EE server is mostly the same as patching in a CE deployment.

Naturally, we don't support building the EE server from source, so the finer-grained source level patching is unavailable.

However, as there is a clear need for separate patches for Enterprise and Community deployments, we do support a separate patches-ee hierarchy. This hierarchy should look like this:

project-client
 ├── patches-ee
 │ ├── server
 │ └── solution
 └── ...

When patching, the contents of patches-ee/server will be applied on top of the contents of the folder set in deploy.dest.server. Similarly, the patches in patches-ee/solution will apply to deploy.dest.solution .

Targets The following ee-specific targets are available

Deploy the full system (deploy-ee-all)

$ ant -Dproject=bare38 -Denv=pedro deploy-ee-all

This will deploy both the server patches and the solution. See the targets below for details.

Deploy the solution (deploy-ee-solution)

$ ant -Dproject=bare38 -Denv=pedro deploy-ee-solution

This will deploy just the solution. Although the exception of applying the patches-ee patches, this is identical to the deploy-solution target.

Deploy the server (deploy-ee-server)

$ ant -Dproject=bare38 -Denv=pedro deploy-ee-server

This will just deploy the server patches to an existing EE installation. Note that, unlike the deploy-server target, this doesn't deploy the complete server, only the patches!

Motivation

CBF - Community Build Framework - is an amazing tool to maintain several projects, compile Pentaho and even doing remote deploys using rsync.

However, there are cases where rsync is not the best way to maintain a server installation. At Mozilla, IT uses Puppet to maintain everything related to automated server maintenance. The way CBF is structured allows us to define rules to automate solution checkouts, but we need to automate the installation of 3 extra bits:

  • Pentaho webapp
  • Pentaho solution (the system/ and admin/ directories)
  • Administration / Enterprise console

Approach

The cluster uses RHEL6 distribution, so we chose to pack the distributions using rpm format. This should work for any rpm-based distribution. If someone wants to do the same for deb or something else, just get in touch or simply fork CBF on github and send a pull request.

Building RPMs through CBF

CBF has now two extra options (be sure to grab the latest version from github):

create-rpm-dist - Creates a tgz file with Pentaho, Pentaho-style and ROOT folders - to be used with a puppet script

create-rpms - Creates rpms based on the specs found on the config/rpm folder for the solution

In order to make it usable, we need to create a directory called rpm under config, and place there the .spec files for the rpm's we're creating. We will also need a specific makefile.

This is the structure:

project-metrics
├── config
│   ├── build-ee-pedro.properties
│   ├── build-pedro.properties
│   ├── build.properties
│   └── rpm
│       ├── makefile
│       ├── webdetails-pentaho-adminconsole.spec
│       ├── webdetails-pentaho-adminconsole-stage.spec
│       ├── webdetails-pentaho-solution.spec
│       ├── webdetails-pentaho-solution-stage.spec
│       ├── webdetails-pentaho-webapp.spec
│       └── webdetails-pentaho-webapp-stage.spec
├── etl
├── patches
│   ├── project-metrics
│   ├── target-build
│   └── target-dist
├── patches-ee
│   ├── enterprise-console
│   ├── server
│   └── solution
└── solution -> ../project-metrics-solution

You'll see that I'm building here 6 RPMs, for a production and stage environment.

You can download the entire set of files from here.

Here's a sample of the file webdetails-pentaho-solution.spec:

Name:  Webdetails-Pentaho-Webapp
Version: @VERSION@
Release: 1%{?dist}
Summary: Pentaho webapp customized for Mozilla metrics
Group:  Applications/Engineering
License: MPL-1.0
Source:     webdetails-pentaho.tgz


%description

%prep
rm -rf %{buildroot}
%build
tar zxvf ../SOURCES/webdetails-pentaho.tgz 

%install
mkdir %{buildroot}
make DESTDIR=%{buildroot} install

%clean
rm -rf %{buildroot}

%files
%defattr(-,root,root,-)
%doc

/opt/pentaho/pentaho-server/server/webapps/pentaho
/opt/pentaho/pentaho-server/server/webapps/pentaho-style
/opt/pentaho/pentaho-server/server/webapps/ROOT

%changelog

And here's the makefile that is called by the specs. I chose to install the Pentaho server under /opt/pentaho/, but this can be changed to any desired path:

all: install
    
install:
 mkdir $(DESTDIR)/opt
 mkdir $(DESTDIR)/opt/pentaho
 mkdir $(DESTDIR)/opt/pentaho/pentaho-server
 mkdir $(DESTDIR)/opt/pentaho/pentaho-server/server/
 mkdir $(DESTDIR)/opt/pentaho/pentaho-server/server/webapps
 cp -r pentaho $(DESTDIR)/opt/pentaho/pentaho-server/server/webapps/pentaho
 cp -r pentaho-style $(DESTDIR)/opt/pentaho/pentaho-server/server/webapps/pentaho-style    
 cp -r ROOT $(DESTDIR)/opt/pentaho/pentaho-server/server/webapps/ROOT    

install-solution:
 mkdir $(DESTDIR)/opt
 mkdir $(DESTDIR)/opt/pentaho
 mkdir $(DESTDIR)/opt/pentaho/pentaho-solution
 cp *.* $(DESTDIR)/opt/pentaho/pentaho-solution/
 cp -r system $(DESTDIR)/opt/pentaho/pentaho-solution/system
 cp -r admin $(DESTDIR)/opt/pentaho/pentaho-solution/admin


install-stage:
 mkdir $(DESTDIR)/opt
 mkdir $(DESTDIR)/opt/pentaho
 mkdir $(DESTDIR)/opt/pentaho/pentaho-server-stage
 mkdir $(DESTDIR)/opt/pentaho/pentaho-server-stage/server  
 mkdir $(DESTDIR)/opt/pentaho/pentaho-server-stage/server/webapps
 cp -r pentaho $(DESTDIR)/opt/pentaho/pentaho-server-stage/server/webapps/pentaho
 cp -r pentaho-style $(DESTDIR)/opt/pentaho/pentaho-server-stage/server/webapps/pentaho-style    
 cp -r ROOT $(DESTDIR)/opt/pentaho/pentaho-server-stage/server/webapps/ROOT    

install-solution-stage:
 mkdir $(DESTDIR)/opt
 mkdir $(DESTDIR)/opt/pentaho
 mkdir $(DESTDIR)/opt/pentaho/pentaho-solution-stage
 cp *.* $(DESTDIR)/opt/pentaho/pentaho-solution-stage/ 
 cp -r system $(DESTDIR)/opt/pentaho/pentaho-solution-stage/system
 cp -r admin $(DESTDIR)/opt/pentaho/pentaho-solution-stage/admin

Dependencies

Note that it depends on having rpmbuild installed on your system. Would work on Linux and OSx, and even eventually on Windows with Cygwin.

After running the create-rpms ant target, you should get the RPMs in the dist/rpm/RPMS/ directory.

Next Steps

You probably know what you want to do with the RPMs. You can either install them directly, or if you're using Puppet you may want to set up a yum repository. There are lots of tutorials on how to do this, here's one of them.

Packaging Enterprise Edition

By default, this targets know how to handle the CE version. We can change a few options and tell it to pack the Enterprise Edition, just by setting some extra options, pointing to a different location where we unpacked the EE version:

######################################
## EE BASED RPM BUILD
######################################

rpm.source.webapp = /home/pedro/tex/pentaho/ee/4.8/biserver-ee/tomcat/webapps/
rpm.source.solution = /home/pedro/tex/pentaho/ee/4.8/biserver-ee/pentaho-solutions/
rpm.source.administration-console = /home/pedro/tex/pentaho/ee/4.8/enterprise-console

Quickstart

One of the most difficult things in CBF is to get it up and running. In order to make things simpler, we prepared a quickstart bundle.

Download the quickstart bundle here and unzip it.

This will create the following structure:

cbf-quickstart
├── build.xml
├── pentaho-3.8
│   └── SVN_CHECKOUT_PENTAHO_HERE
└── project-client
    ├── config
    │   ├── build-pedro.properties
    │   └── build.properties
    ├── patches
    │   └── target-dist
    │       └── server
    │           └── webapps
    │               └── pentaho
    │                   └── WEB-INF
    │                       ├── classes
    │                       │   └── log4j.xml
    │                       └── web.xml
    └── solution
        ├── ClientSolution
        ├── cdf
        │   ├── components
        │   │   ├── jfreechart.xaction
        │   │   ├── jfreechart.xaction.sql
        │   │   ├── jfreechartdial.xaction
        │   │   ├── jfreechartdial.xaction.sql
        │   │   ├── jpivot.xaction
        │   │   ├── jtable.xaction
        │   │   ├── jtable.xaction.sql
        │   │   ├── kettletransformation.xaction
        │   │   ├── openflashchart.xaction
        │   │   ├── openflashchart.xml
        │   │   ├── timelineeventfeeder.xaction
        │   │   ├── timelinefeeder.xaction
        │   │   ├── timelinefeeder.xaction.sql
        │   │   ├── tmp_jfreechart.xaction
        │   │   └── traffic.xaction
        │   └── index.xml
        ├── index.properties
        └── index.xml

Create your environment's configuration file

Copy build-pedro.properties to build-yourname.properties and edit it according to your computer.

Download apache tomcat

Download the .zip or .tgz file from Apache page and uncompress it into the CBF Quickstart directory.

Checkout Pentaho from SVN

After checking this out, checkout Pentaho from SVN:

svn co svn://source.pentaho.org/svnroot/bi-platform-v2/tags/3.8.0-stable pentaho-3.8

Run CBF

Use the command:

ant -Dproject=client -Denv=yourname dist-clean all run

FAQ

What does CBF mean?

CBF stands for Community Build Framework.

Build complains about missing xmltask

You need to download xmltask.jar and put in your ant lib directory (e.g.: /usr/share/ant/).

Is CBF free?

CBF is licensed under the MPLv2 license.

The build fails with a lot of errors on junit. How do I fix it?

Same as with xmltask: you need to put junit.jar in your ant lib dir. This is a requirement since Pentaho 3.8.

Why should I prefer CBF over a pre-compiled version of the Pentaho server?

See the About page and you will find out!

Recommend this article:

Registered in Portugal. NIF 508 528 283