Openshift ingress setup [UPDATE]

Posted by

A lot of openshift user want to use some other ports to reach openshift services inside the openshift cluster then 80 or 443. I describe here how you can setup a so called ingress router and some pitfalls which could happen.

First of all there is the official Openshift Container Platform document for this topic.

Getting Traffic into the Cluster

There is also a github pull request to make the documentation better.

https://github.com/openshift/openshift-docs/pull/3342

Pre-requirement

What Description Example Value
External IP / Range To be able to connect to the external IP you MUST use another IP range then the one which is in use for Service and Nodes in the Openshift SDN 10.20.10.11/32 or 10.20.20.0/25
Firewall port open If you have a external firewall in front of OSCP you MUST open the port to be able to connect to this port Dynamic assigned by openshift. see ServicesNodePortRange
Cluster admin right Due to the fact that you will need to give a service account the hostaccess rights you will need cluster-admin right or someone which have this.
Dedicated Node You need to decide if you want to run the ingress route on dedicated nodes or on the OSCP Router

Introduction

The setup looks like this.

oscp-ingress-001

The ipfailover is the same as described in High Availability for the cluster him self. Due to this fact the concept is well tested and works quite nice.

The DC postgresql-ingress must be from Type LoadBalancer . I will use the configs from the upcoming documentation as described above.

Setup

Master config

You will need to add the decided network-range in the master-config.yml

networkConfig:
  ingressIPNetworkCIDR: 10.20.20.0/25

You can also add this value in the ansible inventory file.

openshift_master_ingress_ip_network_cidr=10.20.20.0/25

UPDATE: the commit d9fe14e9b53590d7949cbdd53cedb89bbc0ee037 have introduced the Variable openshift_node_port_range for the node port range.

You are be able to add this line into the ansible inventory file which opens then the ports on the nodes.

openshift_node_port_rang=30000-32000

Which ever way you go, you will need to restart the master(s).

New project

Now first of all login to a Openshift Container Plattform and create a project where you can setup the ingress router.

oc login ....
oc new-project my-ingress

You are now in the project my-ingress.

Now create the postgresql app

oc new-app postgresql-ephemeral

Serviceaccount

Due to the fact that the keepalived, the software behind ipfailover, must bind to the host/node port you will need to add the privilege to the new ipfailover serviceaccount.

$ oc create serviceaccount ipfailover
$ oc adm policy add-scc-to-user hostnetwork \
    system:serviceaccount:my-ingress:ipfailover

Services

To be able to access the postgresql from outsite the cluster you need a new service additional to the one which create the new-app generator.

Now we can expose the new service

$ oc expose dc postgresql --name postgresql-ingress --type=LoadBalancer

At this point the openshift will use a random port to listen on node.

You can get the port via this command.

oc get svc postgresql-ingress -o jsonpath='{.spec.ports[?(@)].nodePort}'

IPFailover

The last step is now to create the ipfailover service which then run the ip up.

oadm ipfailover ipf-ha-postgresql \
 --replicas=1 --selector="region=infra-ingress" \ 
 --virtual-ips=10.20.20.100 --interface=eth1 --watch-port=<THE_NODEPORT_INGRESS_SVC> \
 --service-account=ipfailover —create

Now cross the fingers and see if the ip is up 😉

The workflow is now like this.

oscp-ingress-002

Pitfalls

Node Selector

To be able to run in one project pods on different nodes you will need to set the annotation openshift.io/node-selector to a empty value.

$ oc patch ns my-ingress \
    -p '{"metadata": {"annotations": {"openshift.io/node-selector": ""}}}'

It is your decision where the ingress pods ( external IP ) should be setuped.

Connection errors

When the connection was successfully initiated you can scale down the ipfaiover pods and the db connection is still alive, why?

Well the key is the postgresql-ingress as long as this service is up you will have a valid connection.

oscp-ingress-003

Nodeports lower then 30000

When you want to use a another nodportrange then you will need to change the servicesNodePortRange on all masters in the master-config.yml

You can also hire me for this or any further topics.

Kommentar verfassen

Trage deine Daten unten ein oder klicke ein Icon um dich einzuloggen:

WordPress.com-Logo

Du kommentierst mit Deinem WordPress.com-Konto. Abmelden / Ändern )

Twitter-Bild

Du kommentierst mit Deinem Twitter-Konto. Abmelden / Ändern )

Facebook-Foto

Du kommentierst mit Deinem Facebook-Konto. Abmelden / Ändern )

Google+ Foto

Du kommentierst mit Deinem Google+-Konto. Abmelden / Ändern )

Verbinde mit %s