Creating a CloudShare Team Labs Environment with SAP HANA, SAP Data Services and Hortonworks Hadoop

BIG DATA AND IN-MEMORY DATABASE

Nowadays when you are looking for Big Data techniques you get in touch with Hadoop. If you are googling for an In-Memory Database you will find SAP HANA. These are two actual topics which are on top of the Gartner Hype Cycle of Emerging Technologies (Gartner, 2014).

That’s one of the reason why we from IT-Logix wanted to create a test environment which all of our consultants can access easily. But technologies like SAP HANA needs lots of HDD space, let alone of memory. The minimum memory requirement even just for testing SAP HANA is 24 GB (SAP, 2014). As we don’t need a production system which is always up and running we were looking for a cheaper solution compared to buying new hardware and host the new test system on premise.

REQUIREMENTS LED US TO CLOUDSHARE TEAM LABS

Cloudshare provides us exactly what we were looking for: A flexible platform to host virtual machines at a very low cost due to the fact that the systems get suspended after a certain period of user inactivity. We already used the Cloudshare offering “ProPlus” to host the SAP BusinessObjects BI platform (see Raphael’s blog here: http://rbranger.wordpress.com/2012/04/30/testing-bo-bi-4-x-using-the-cloud/ ) Within CloudShare Pro the maximum memory per virtual machine is limited to 16GB. Further we need to install operating systems like SUSE Linux Enterprise Server 11 or Red Hat 6.5 which are not available as preconfigured templates.. All these requirements and even more will easily be covered by CloudShare Team Labs, so we decided to use this for our new test environment.
Let’s start configuring the Environment.

 

Overview Virtual Machines Overview Virtual Machines

HORTONWORKS HADOOP SANDBOX

Hortonworks Hadoop is a preconfigured virtual machine which you can download 100% free. Besides the Apache Hadoop Database it includes numerous tools for data management, data access, data governance & integration, security and operations. A perfect system to learn and test Apache Hadoop.
Now we will start to implement the Sandbox:

1. Download the Sandbox for VMWare from here
2. Import the Sandbox to your local VMWare Workstation, ESX, vCenterServer or Fusion software
3. Follow the instruction for uploading your own image to CloudShare: Link

SAP Data Services

SAP Data Services is a really powerful ETL tool. To work with this tool you’ll need also SAP IPS or BusinessObjects. You need a SAP Partner Agreement or customer contract to download it from the SAP Service Marketplace. To install and configure SAP Data Services on CloudShare Team Labs we decided to first install it on our local system or e.g. on a VMWare ESXi Server. This way we are free to choose our own operating system and configure it from scratch. In our case we are working with VMWare Workspace. Once we are finished, we’ll export the virtual machine and follow again the instructions for uploading your own image to CloudShare (Link).

SAP Data Services can be installed on Red Hat 6.5 or CentOS 6.5 with a little hack (and of course other operating systems, but in this case we wanted to play around with the Red Hat version) . We’ll show you both alternatives.

1. Download the Image of Red Hat 6.5 or CentOS 6.5
2. Create a new VM within VMWare Workstation and load the downloaded Image of the OS.
3. The installation will be done automatically. When it finished, login with root user and do following steps to install SAP IPS:

Create a user for DS and change the password:
adduser boadmin
passwd boadmin

Create a folder for the installation:
cd /opt
mkdir bobj
mkdir install

Set the rights on the created folders:
chown boadmin /opt/bobj
chmod 755 /opt/bobj

Customize the hostfile, so that the local system knows his internal ip-address (enter also the full qualified domainname):
vi /etc/hosts
192.168.xx.xx itxds itxds.local

Customize the selinux file (Security-Enhanced Linux):
vi /etc/sysconfig/selinux
SELINUX=PERMISSIVE

Add following local variables:
vi /etc/sysconfig/i18n
LANG=”en_US.UTF-8″
export LC_ALL=”en_US.UTF-8″

Now, if you are using Red Hat 6.5, you need to register it. Otherwise you could not use yum to download packages (not needed for CentOS!)
rhn_register

Install following packages:
yum install libXext.i686 libXext-devel.i686 libXext-devel.x86_64 compat-libstdc++-33-3.2.3-69.el6.i686 libstdc++.i686

If you are using CentOS, now you need to do a little hack. Open the rehat-release file and change the Text (not needed for Red Hat!):
/etc/redhat-release
Red Hat Enterprise Linux Server release 6.5 (Santiago)

Now download SAP IPS e.g with wget from your FTP server and unzip
cd /opt/install
wget –ftp-user=XXX –ftp-password=XXX ftp://IPS41.ZIP
unzip IPS41.ZIP

Before we can start the installation, we should change to the created user
su boadmin

Start the installation
cd /opt/install/DATA_UNITS/IPS_lnx
./setup.sh

Now follow the installation wizard. When the installation process finished, change again to the root user and execute the following script:
/opt/bobj//sap_bobj/init/setupinit.sh

Last but not least, you should open the port 8080
vi /etc/sysconfig/iptables
-A INPUT -m state –state NEW -m tcp -p tcp –dport 8080 -j ACCEPT
service iptables restart

4. You could test SAP IPS by open following URL: http://yourdomain.com:8080/BOE/CMC
5. When this works fine, we’re ready to install SAP Data Services. We will use the SQL Anywhere Database Server which comes with SAP IPS for the Repositories. Therefore you should execute the following steps:

Open the SQL Anywhere port 2638
vi /etc/sysconfig/iptables
-A INPUT -m state –state NEW -m tcp -p tcp –dport 2638 -j ACCEPT
service iptables restart

Check the name of the Database. The name of the Database should be something like “BI4_XXXXXX”
ps –ef | grep sql

Download Sybase Central from here and create a new Connection with the following data:
User-ID: dba
Password: password (from IPS install wizard)
Host: 192.168.XX.XX (replace with IP of your Server)
Port: 2638
Servername: BI4_XXXXXX (replace with DB number from the last step)
Databasename: BI4_CMS

Now connect to your Database and choose on the left panel of Sybase Central “Create a new Database on Server BI4_XXXXXX”. Name the Database e.g. “DSRepo

Download SAP DS e.g with wget from you ftp server and unzip
cd /opt/install
wget ftp://DS.ZIP
unzip DS.ZIP

Start the setup
./setup

Follow the wizard. Choose the same install directory as for SAP IPS. When the wizard asks you for the SQL Anywhere driver, type following path:
/opt/bobj/sqlanywhere/lib64/libdbodbc12_n.so

6. Last but not least, follow the instruction for uploading your own image to CloudShare: Link

Configure OS for SAP HANA

HANA is new a database technology of SAP. It uses In-Memory-Technology which is significantly faster compared to legacy disk based databases. To install and configure it on CloudShare Team Labs we recommend to configure the operating system locally, then upload the VM to the CloudShare Environment and install SAP HANA directly on the system in Cloudshare. The reason for this procedure is that our local systems don’t have 24GB of free memory to install HANA. Moreover, it takes much more time to upload an image which is about 15GB.
The newest HANA revision 80 supports also Red Hat beside SLES. But we decided to use SLES because there isn’t much experience with HANA on Red Hat right now.

1. Download SUSE Linux Enterprise Server 11 SP3
2. Create a new VM within VMWare Workstation and load the downloaded Image of the OS.
3. The installation will be done automatically. When it finished, login with root user and do following steps:

Change Hostfile
vi /etc/hosts
192.168.xx.xx %HOSTNAME%

Disable Firewall
yast firewall

Now install packages needed for SAP HANA. Therefore, load the SLES11 Installation DVD1 to the virtual CD-Drive
zypper in gtk2 java-1_6_0-ibm libicu Mozillaxulrunner* ntp syslog-ng tcsh libssh2-1 expect autoyast2installation yast2-ncurses python libgcc_s1 libstdc++6

Note: If you are using SLES 11 SP2 you need to install gcc47-runtime manually.
Therefore check SA note 2001528

Set SLES kernel parameter
vi /etc/sysctl.conf
net.ipv4.tcp_slow_start_after_idle = 0
kernel.shmmni = 4096

Set Environment Variable
export IDSPISPOPD=”1″

Set NTP Server:
vi /etc/ntp.conf
server %your nationl timeserver%

/etc/init.d/ntp restart
chkconfig ntp on
ntpq –p

4. Now follow the instruction for uploading your own image to CloudShare: Link

Configure the Network

Before we can install SAP HANA on the just uploaded SLES OS, we need to configure the network. Prior to that we cannot download the HANA Setup.

 

Edit Environment Edit Environment

When all VMs are up and running on CloudShare TeamLabs, we need to configure the network. Therefore click on Edit Environment / Edit Networks in the right Panel of your Environment:

Now you’ll see a network topology. Select the root node and define the Subnet and Gateway IP. If you select the Advanced Mode, you could define whether outgoing and internal network is allowed:

 

Network Topology Network Topology

After configuring the root node, select the other VMs and assign them to the Subnet. If you like to work to work with our local installation of SAP HANA Studio, you have to select Public IP to make your VM accessible from outside the Cloud. For SAP HANA you need to open Port 30015 in order to connect with your local SAP HANA Studio. Port 8000 is for the XS Engine and Port 22 is SSH.

Note: Not all ports could be opened. Due to security reasons, CloudShare only allow some ports to open e.g. 22, 80, 443, 8000, 8080, 30015. All ports which are not entered here, are closed – so you don’t need a necessary a firewall on the OS.

 

Network Details Network Details

When you assigned all VMs to the network we need to configure the network cards of the VMs.

1. Open the SAP HANA VM in GUI mode and login
2. Open the Startmenu and select Network
3. Select the “Overview” Tab, select the Gigabit Ehternet Controller and click on Edit
4. Choose statically assigned IP Address and assign the IP Address you defined within CloudShare:

 

SLES - Network Card Setup 1 SLES – Network Card Setup 1

5. Define the Name Server in the “Hostname/DNS” Tab:

 

SLES - Network Card Setup 2 SLES – Network Card Setup 2

6. Define the default Gateway:

 

SLES - Network Card Setup 3 SLES – Network Card Setup 3

Repeat this steps for the other two VMs and you’re done with the network configuration.

Install SAP HANA

1. When the image is successfully uploaded and configured within the CloudShare Environment, execute the following steps to install HANA:

Download SAP HANA Database Setup and SAPCAR (e.g. upload with WinSCP or WGET from your ftp server):
wget ftp://yourserver/IMDB_SERVER100.SAR
wget ftp://yourserver/SAPCAR

Use SAPCAR to extract the HANA Setup:
SAPCAR –xvf IMDB_SERVER100.SAR

Start Setup
./hdbinst

2. Follow the setup wizard.