Installing Component Pack 6.0.0.6

Berlin, October 16-17 2018
Installing Component
Pack 6.0.0.6
Martti Garden
Roberto Boccadoro

PLATINUM SPONSORS
GOLD SPONSORS
BRONZE SPONSORS
SILVER SPONSORS
SPEEDSPONSORING BEER SPONSOR

Social Connections 14 Berlin, October 16-17 2018
Who are we
Martti Garden
IBM Technical Leader Social Europe
@mgarden
http://socialibmer.com/
Roberto Boccadoro
Sr. Consultant at ELD Engineering
IBM Champion
@robboc59
http://robertoboccadoro.com

Who are we
Brendan Furey
Advisory Software Engineer –
IBM Connections Component Pack
Conall O’Cofaigh
Advisory Software Engineer –
IBM Connections Component Pack

Getting the prerequisites ready
• Docker
• Kubernetes
• Helm
• Docker Registry
• Persistent Volumes

Installing Docker 17.03 (on each machine)
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
yum-config-manager --disable docker*
yum-config-manager --enable docker-ce-stable
yum install -y --setopt=obsoletes=0 docker-ce-17.03*
yum makecache fast
sudo systemctl start docker
sudo systemctl enable docker.service
yum-config-manager --disable docker*

PoC: Configure Docker with the devicemapper storage driver (loop-lvm)
– on each server
sudo systemctl stop docker
vi /etc/docker/daemon.json
add:
{
"storage-driver": "devicemapper"
}
sudo systemctl start docker
Verify by docker info

PoC: Configure Docker with the devicemapper storage driver (loop-lvm)
– on each server
swapoff -a
vi /etc/fstab
Comment out /dev/mapper/cl-swap swap swap defaults 0 0
mount -a

Install kubeadm, kubelet, and kubectl (on each server)
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF

The setenforce 0 command disables SELinux to allow containers to access the host file system (required by pod
networks, for example)
setenforce 0
yum install -y kubelet-1.11.1* kubeadm-1.11.1* kubectl-1.11.1*
systemctl enable kubelet && systemctl start kubelet
yum-config-manager --disable kubernetes*

Some users on RHEL/CentOS 7 have reported issues with traffic being routed incorrectly due to iptables being
bypassed. To avoid this problem, run the following commands to ensure that net.bridge.bridge-nf-call-iptables is set to 1
in your sysctl config:
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system

Initializing Master (on Master)
In this example we use Calico as pod network addon:
kubeadm init --kubernetes-version=v1.11.1 --pod-network-cidr=192.168.0.0/16
Make sure to copy out the join command at the end, as we will need it later!

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Install a pod network add-on (here Calico) so that your pods can communicate with each other.
kubectl apply -f https://docs.projectcalico.org/v3.1/getting-
started/kubernetes/installation/hosted/rbac-kdd.yaml
kubectl apply -f https://docs.projectcalico.org/v3.1/getting-
started/kubernetes/installation/hosted/kubernetes-datastore/calico-
networking/1.7/calico.yaml

Join Workers (on Worker Nodes)
Remember the join command that we copied? We now run it on both Worker Nodes.
kubeadm join 159.8.241.236:6443 --token hslnj3.4c1s75477654flt0 --discovery-token-ca-
cert-hash sha256:de3422452417c652145235747474746540ac8297e2eb5

On the master you can now test if the nodes have been joined successfully:
kubectl get nodes

Now we copy the Master configuration to the Worker nodes for kubectl
mkdir -p $HOME/.kube
scp root@159.8.241.236:$HOME/.kube/config $HOME/.kube
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Installing Helm (on Master)
wget https://storage.googleapis.com/kubernetes-helm/helm-v2.11.0-linux-amd64.tar.gz
tar -zxvf helm-v2.11.0-linux-amd64.tar.gz
sudo mv linux-amd64/helm /usr/local/bin/helm
helm init
kubectl create clusterrolebinding add-on-cluster-admin --clusterrole=cluster-admin --
serviceaccount=kube-system:default
sudo rm -f helm-v2.11.0-linux-amd64.tar.gz

Installing Helm (on Master)
Test environment on master by checking that everything is running
kubectl get pods -n kube-system

Create Connections Namespace (on Master)
kubectl create namespace connections

Installing Docker registry (on Master)
Create directories:
mkdir /docker-registry
mkdir /docker-registry/{auth,certs,registry}
Create password file:
docker run --entrypoint htpasswd registry:2 -Bbn admin mypassword > /docker-
registry/auth/htpasswd

Create self signed certs:
openssl req -newkey rsa:4096 -nodes -sha256 -keyout key.pem -x509 -days 3650 -out cert.pem
Copy cert and key to docker directory:
cp key.pem cert.pem /docker-registry/certs

Create directories on all machines in cluster:
mkdir /etc/docker/certs.d
mkdir
/etc/docker/certs.d/soc.ibmcollabcloud.com:500
0/
Copy cert to docker dir:
cp cert.pem
/etc/docker/certs.d/soc.ibmcollabcloud.com:500
0/ca.crt
SCP the cert from the docker registry machine to all other machines in
the kubernetes cluster:
scp cert.pem
soc1.ibmcollabcloud.com:/etc/docker/certs.d/soc.ibmc
ollabcloud.com:5000/ca.crt
scp cert.pem
soc2.ibmcollabcloud.com:/etc/docker/certs.d/soc.ibmc
ollabcloud.com:5000/ca.crt

Create registry:
docker run -d -p 5000:5000 --restart=always --name registry
-v /docker-registry/auth:/auth -v /docker-
registry/certs:/certs -v /docker-
registry/registry:/var/lib/registry -e
"REGISTRY_AUTH=htpasswd" -e
"REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" -e
"REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd" -e
"REGISTRY_HTTP_TLS_CERTIFICATE=/certs/cert.pem" -e
"REGISTRY_HTTP_TLS_KEY=/certs/key.pem" registry:2
Verify:
docker login -u admin -p mypassword
soc.ibmcollabcloud.com:5000
Create image pull secret
kubectl create secret docker-registry myregkey -n
connections --docker-server=soc.ibmcollabcloud.com:5000 --
docker-username=admin --docker-password=mypassword

Create persistent volumes (on Master / NFS Server)
sudo mkdir -p /pv-connections/esdata-{0,1,2}
sudo mkdir -p /pv-connections/esbackup
sudo mkdir -p /pv-connections/customizations
sudo mkdir -p /pv-connections/mongo-node-
{0,1,2}/data/db
sudo mkdir -p /pv-connections/solr-data-solr-
{0,1,2}
sudo mkdir -p /pv-connections/zookeeper-data-
zookeeper-{0,1,2}
sudo chmod -R 777 /pv-connections
cd
/root/cp6006/microservices_connections/hybridclou
d/support/
sudo bash nfsSetup.sh
Install persistent volumes using Helm
helm install --name=connections-volumes
/root/cp6006/microservices_connections/hybridclou
d/helmbuilds/connections-persistent-storage-nfs-
0.1.0.tgz --set nfs.server=159.8.241.236

Labeling and tainting worker nodes for Elasticsearch (on Master)
kubectl get nodes
kubectl label nodes
soc2.ibmcollabcloud.com
type=infrastructure –overwrite
kubectl taint nodes
soc2.ibmcollabcloud.com
dedicated=infrastructure:NoSchedule --
overwrite

Pushing the images to the Docker registry (on Master)
cd
/root/cp6006/microservices_connections/
hybridcloud/support
./setupImages.sh -dr
soc.ibmcollabcloud.com:5000 -u admin -p
mypassword -st
customizer,elasticsearch,orientme

Bootstrapping the Kubernetes cluster (on Master)
helm install --name=bootstrap
/root/cp6006/microservices_connections/hybridcloud/h
elmbuilds/bootstrap-0.1.0-20180924-133245.tgz --set
image.repository="soc.ibmcollabcloud.com:5000/conne
ctions",env.set_ic_admin_user=wasadmin,env.set_ic_a
dmin_password=ibm4MBI4,env.set_ic_internal=con.ib
mcollabcloud.com,env.set_master_ip=159.8.241.236,e
nv.set_elasticsearch_ca_password=mypassword,env.s
et_elasticsearch_key_password=mypassword,env.set_
redis_secret=mypassword,env.set_search_secret=myp
assword,env.set_solr_secret=mypassword
kubectl get pods -n connections -a | grep bootstrap

Installing the Component Pack

Component Pack
Helm Chart Orient Me Customizer Elasticsearch
bootstrap ✔ ✔ ✔
connections-env ✔ ✔ ✔
infrastructure ✔ ✔
mw-proxy ✔
elasticsearch ✔
orientme ✔

Installing the Component Pack's connections-env (on master)
helm install --name=connections-env
/root/cp6006/microservices_connections/hybridcloud/helmbuilds/connections-env-0.1.40-
20180919-173326.tgz --set
createSecret=false,ic.host=con.ibmcollabcloud.com,ic.internal=con.ibmcollabcloud.com
helm list

Installing the Component Pack's infrastructure (on master)
helm install --name=infrastructure
/root/cp6006/microservices_connections/hybridcloud/helmbuilds/infrastructure-0.1.0-20180925-
030258.tgz --set
global.onPrem=true,global.image.repository=soc.ibmcollabcloud.com:5000/connections,mongodb.c
reateSecret=false,appregistry-service.deploymentType=hybrid_cloud

Installing the Component Pack's Orient Me (on master)
helm install --name=orientme
/root/cp6006/microservices_connections/hybridcloud/helmbuilds/orientme-0.1.0-20180925-
030334.tgz --set
global.onPrem=true,global.image.repository=soc.ibmcollabcloud.com:5000/connections,orient-
web-client.service.nodePort=30001,itm-services.service.nodePort=31100,mail-
service.service.nodePort=32721,community-suggestions.service.nodePort=32200

Installing the Component Pack's ElasticSearch (on master)
helm install --name=elasticsearch
/root/cp6006/microservices_connections/hybridcloud/helmbuilds/elasticsearch-0.1.0-20180921-
115419.tgz --set
image.repository=soc.ibmcollabcloud.com:5000/connections,nodeAffinityRequired=true

Installing the Component Pack's Customizer (on master)
helm install --name=mw-proxy
/root/cp6006/microservices_connections/hybridcloud/helmbuilds/mw-proxy-0.1.0-20180924-
103122.tgz --set
image.repository=soc.ibmcollabcloud.com:5000/connections,deploymentType=hybrid_cloud

Installing the Dashboards for monitoring and logging (on master)
mkdir /opt/kubernetes-dashboard
openssl req -nodes -new -x509 -keyout /opt/kubernetes-
dashboard/dashboard.key -out /opt/kubernetes-
dashboard/dashboard.crt -subj "/CN=dashboard„
kubectl create secret generic kubernetes-dashboard-certs --from-
file=/opt/kubernetes-dashboard -n kube-system
kubectl apply -f
https://raw.githubusercontent.com/kubernetes/dashboard/master/src
/deploy/recommended/kubernetes-dashboard.yam
lkubectl apply -f
/root/cp6006/microservices_connections/hybridcloud/support/dashbo
ard-admin.yaml
kubectl patch svc kubernetes-dashboard -n kube-system -p
'{"spec":{"type": "NodePort"}}‘
kubectl create -f
https://raw.githubusercontent.com/kubernetes/heapster/master/depl
oy/kube-config/influxdb/grafana.yaml
kubectl create -f
oy/kube-config/influxdb/heapster.yaml
kubectl create -f
oy/kube-config/influxdb/influxdb.yaml
kubectl create -f
oy/kube-config/rbac/heapster-rbac.yaml
nohup kubectl proxy --address=159.8.241.236 -p 443 --accept-
hosts='^*$' &

Installing the Component Pack's Sanity Dashboard (on master)
helm install --name=sanity
/root/cp6006/microservices_connections/hybridcl
oud/helmbuilds/sanity-0.1.8-20180924-121014.tgz
--set
image.repository=soc.ibmcollabcloud.com:5000/co
nnections
helm install --name=sanity-watcher
/root/cp6006/microservices_connections/hybridcl
oud/helmbuilds/sanity-watcher-0.1.0-20180830-
052154.tgz --set
image.repository=soc.ibmcollabcloud.com:5000/co
nnections
export NODE_PORT=$(kubectl get --namespace
connections -o
jsonpath="{.spec.ports[0].nodePort}" services
sanity)
export NODE_IP=$(kubectl get nodes --namespace
connections -o
jsonpath="{.items[0].status.addresses[0].addres
s}")
echo http://$NODE_IP:$NODE_PORT

Installing the Component Pack's Elastic Stack (on master)
helm install --name=elasticstack
/root/cp6006/microservices_connections/hybridcloud/helmbuilds/elasticstack-0.1.0-20180925-
030346.tgz --set
global.onPrem=true,global.image.repository=soc.ibmcollabcloud.com:5000/connections

Configuring the components

Orient Me

Orient Me
Edit httpd.conf con Connections Server
Uncomment:
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule proxy_ftp_module modules/mod_proxy_ftp.so LoadModule proxy_http_module modules/mod_proxy_http.so
Before the LoadModule ibm_ssl_module modules/mod_ibm_ssl.so statement and between the <VirtualHost *:443> and </VirtualHost> statements add:
ProxyPreserveHost On
ProxyPass "/social" "http://soc.ibmcollabcloud.com:30001/social"
ProxyPassReverse "/social" "http://soc.ibmcollabcloud.com:30001/social"
ProxyPass "/itm" "http://soc.ibmcollabcloud.com:31100/itm"
ProxyPassReverse "/itm" http://soc.ibmcollabcloud.com:31100/itm
ProxyPass "/community_suggestions/api/recommend/communities" http://soc.ibmcollabcloud.com:32200/community_suggestions/api/recommend/communities
ProxyPassReverse "/community_suggestions/api/recommend/communities"
http://soc.ibmcollabcloud.com:32200/community_suggestions/api/recommend/communities
ProxyPass "/appreg" http://soc.ibmcollabcloud.com:30285
ProxyPassReverse "/appreg" http://soc.ibmcollabcloud.com:30285
ProxyPass "/appregistry" "http://soc.ibmcollabcloud.com:32212/appregistry"
ProxyPassReverse "/appregistry" http://soc.ibmcollabcloud.com:32212/appregistry
Restart HTTP Server

Orient Me
Test
http://con.ibmcollabcloud.com/social/views/login.html

Enabling profiles events for Orient Me
Edit TDI/conf/LotusConnections-config/tdi-profiles-config.xml:
Within the tdiConfig section, add a <properties>
<properties>

<property name="com.ibm.lconn.profiles.config.EnableManagerChangeEvent" value="true"/>
<property name="com.ibm.lconn.profiles.config.EnableTDIEventOverride" value="true"/>
</properties>
Then run:
sync_all_dns.bat

Enabling profiles events for Orient Me
In the <properties> section of profiles

<property name="com.ibm.lconn.profiles.config.EnableManagerChangeEvent" value="true"/>
<property name="com.ibm.lconn.profiles.config.EnableTDIEventOverride" value="true"/>

Configuring notifications for the Orient Me homepage
Edit LotusConnections-config.xml and uncomment the OrientMe Service reference:

<sloc:serviceReferenceserviceName="orient„
enabled=“true„
ssl_enabled=“true„
bootstrapHost="con.ibmcollabcloud.com„
bootstrapPort="2809„
clusterName="">
<sloc:href>
<sloc:hrefPathPrefix>/social</sloc:hrefPathPrefix>
<sloc:static href="http://con.ibmcollabcloud.com" ssl_href="https://con.ibmcollabcloud.com" />
<sloc:interService href="https://con.ibmcollabcloud.com" />
</sloc:href>
</sloc:serviceReference>

Enable the actioncenter
<genericProperty name="actioncenter">enabled</genericProperty>
Now is a great time to restart Connections!

Populating the Orient Me home page
kubectl exec -n connections -it $(kubectl get pods -n connections | grep people-migrate | awk '{print $1}') bash
npm run start migrate

Customizer

Customizer (on NGINX HTTP Server)
Install nginx
yum install epel-release
yum install nginx
On Connections machine enable dynamic hosts in LotusConnections-config.xml
<dynamicHosts enabled="true">
<host href="http://soc.ibmcollabcloud.com" ssl_href="https://soc.ibmcollabcloud.com"/>
</dynamicHosts>
Sync nodes & Restart Connections

Customizer (on Master)
kubectl get configmap connections-env -o yaml -n connections | grep customizer-
interservice-host

Create SSL certificates for nginx
mkdir /etc/nginx/ssl
openssl req -x509 -nodes -days 3650 -newkey rsa:2048 -keyout /etc/nginx/ssl/nginx-
selfsigned.key -out /etc/nginx/ssl/nginx-selfsigned.crt

#Change paths and server / port in the server section: worker_processes 1;
vi /etc/nginx/nginx.conf
events {
worker_connections 16384;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
listen 80;
server_name localhost;
root html;
index index.html index.htm;
proxy_pass http://soc.ibmcollabcloud.com:30301;
}
# redirect server error pages to the static page /50x.html
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffers 32 4k;
server {
listen 443 ssl;
server_name 127.0.0.1;
ssl_certificate /etc/nginx/ssl/nginx-selfsigned.crt;
ssl_certificate_key /etc/nginx/ssl/nginx-selfsigned.key;
ssl_session_cache shared:SSL:1m;
ssl_session_timeout 5m;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
location
{
#Points to the master with mw-proxy. Port should be as below
proxy_pass http://soc.ibmcollabcloud.com:30301;
}
}
}
vi /etc/nginx/nginx.conf

Test configuration
nginx –t

Now we need to disable the firewall between nginx and the
kubernetes masters
sudo setsebool -P httpd_can_network_connect
true
Next we set nginx to start automatically
mkdir /etc/systemd/system/nginx.service.d
cat <<EOF >
/etc/systemd/system/nginx.service.d/nofile_l
imit.conf
[Service]
LimitNOFILE=16384
EOF
systemctl daemon-reload

Time to start nginx
sudo systemctl start nginx

Configuring Orient Me to support a reverse-proxy server
kubectl get configmap connections-env -o yaml -n connections | grep ic-homepage-url
kubectl get configmap connections-env -o yaml -n connections | grep ic-host
kubectl get configmap connections-env -o yaml -n connections | grep orient-cnx-host

helm upgrade connections-env
/root/cp6006/microservices_connections/hybridcloud/helmbuilds/connections-env-0.1.40-
20181011-103145.tgz --set
createSecret=false,ic.host=soccxn.ibmcollabcloud.com,ic.internal=con.ibmcollabcloud.com

Now we kill all Pods with updated config configmap
kubectl -n connections delete pods -l app=appregistry-client
kubectl -n connections delete pods -l app=appregistry-service
kubectl -n connections delete pods -l app=community-suggestion
skubectl -n connections delete pods -l app=itm-services
kubectl -n connections delete pods -l app=middleware-graphql
kubectl -n connections delete pods -l app=orient-web-client
kubectl -n connections delete pods -l app=people-migrate

Now we can test if we can reach the app registry:
http://soccxn.ibmcollabcloud.com/appreg

Elasticsearch

Enabling Elasticsearch Metrics to connect to a Component Pack server
cd /root/cp6006/microservices_connections/hybridcloud/support
python config_blue_metrics.py --skipSslCertCheck true --pinkhost soc.ibmcollabcloud.com

Granting access to global Elasticsearch Metrics
In WebSphere go to Applications > Application Types > WebSphere enterprise applications > MetricsUI > Security role
to user/group mapping
Add users to "metrics-report-run role"

Optional: Removing SSL settings that were configured for type-ahead search
Only needed if you had configured QuickResults before.
In the WebSphere Integrated Solutions Console:
Click Security > SSL certificate and key management > Dynamic outbound endpoint SSL
configurations and, for each cluster member, delete the endpoint that begins with "SearchToES".
Click Security > SSL certificate and key management > SSL configurations and delete the setting with
name "ESSearchSSLSettings".
Click Security > SSL certificate and key management > Key stores and certificates and delete the key
store with name "ESCloudKeyStore".

Enabling Elasticsearch Metrics to connect to a Component Pack server
kubectl get secret elasticsearch-secret -n connections -o=jsonpath="{.data['chain-
ca.pem']}" | base64 -d > chain-ca.pem
kubectl get secret elasticsearch-secret -n connections -
o=jsonpath="{.data['elasticsearch-metrics.p12']}" | base64 -d > elasticsearch-
metrics.p12
Copy the certificates to where they
are accessible to DMGR as well as
all Nodes.

Copy Certs to Dmgr01 accessible by all Nodes
cd C:IBMWebSphereAppServerprofilesDmgr01bin
wsadmin -lang jython
execfile('esSecurityAdmin.py')
enableSslForMetrics('C:IBMCPCertselasticsearch-metrics.p12', 'mypassword',
'C:IBMCPCertschain-ca.pem', '30099‘)
execfile('searchAdmin.py')
SearchService.setESQuickResultsBaseUrl("https://soc.ibmcollabcloud.com:30099")
execfile('esSearchAdmin.py')
enableSslForESSearch('C:IBMCPCertselasticsearch-metrics.p12', 'mypassword',
'C:IBMCPCertschain-ca.pem', '30099')
Synchronize the Nodes and then restart the clusters containing the Common and
Search applications

Configuring type-ahead search with Metrics enabled
execfile("searchAdmin.py")
SearchService.createESQuickResultsIndex()

Enabling Elasticsearch based QuickResults
Update the LotusConnections-config.xml in <properties> section
<genericProperty name="quickResultsEnabled">true</genericProperty>
Update the search-config.xml in <properties> section
<property name="quickResults">
<propertyField name='quick.results.elasticsearch.indexing.enabled' value='true'/>
<propertyField name='quick.results.solr.indexing.enabled ' value='false'/>
<propertyField name='quick.results.use.solr.for.queries' value='false'/>
</property>
Synchronize the Nodes and at this time a complete restart including DMGR and
Nodeagents is a good thing.

Deploying Elasticsearch Metrics as your first use of metrics
execfile("metricsEventCapture.py")
switchMetricsToElasticSearch()

ALREADY DONE! 

Installing Component Pack 6.0.0.6

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Installing Component Pack 6.0.0.6

Similar to Installing Component Pack 6.0.0.6 (20)

More from LetsConnect

More from LetsConnect (20)

Recently uploaded

Recently uploaded (20)

Installing Component Pack 6.0.0.6