Welcome To NetTech Solutions

Warm Welcome to Every Visitor. Here we serve best technical support, all the posts here you find will help you solving and facing the day to day problems.
Every one is welcome to comment on our Posts.

Friday, March 14, 2014

Setting Tomcat to run Mutiple instances



Introduction
This document will teach us how to setup Tomcat to run Multiple instances of it.
Intended Audience: System admins.

Instruction to installing Tomcat so that we can run multiple instances.

1. Install the Tomcat files

Download Tomcat and unzip it into an appropriate directory. I usually put it in /opt, so it ends up in a directory called /opt/apache-Tomcat-6.0.29 (6.0.29 being the current version as of this writing), and make a symlink named /opt/tomcat to that directory. When later versions come out, we can unzip them and relink, leaving the older version in case things don’t work out (which rarely if ever happens, but as system admins we are paranoid).

2. Add user for each instance

For each instance of Tomcat we’re going to run, we’ll need to add a user and create Tomcat directory in it that will be CATALINA_BASE. For example, we will add users Tomcat1 and Tomcat2 and create Tomcat directory the CATALINA_BASE will be /home/Tomcat1/Tomcat and /home/Tomcat2/Tomcat2 respectively.

In each users home directory we need the following sub directories: conf, logs, temp, webapps, and work.

We will copy these sub directories from /opt/tomcat in the Tomcat directory of the user. Of course you should tighten up your server.xml a bit.

The webapps directory is where you’ll put the web applications you want to run on the particular instance of Tomcat.

We like to have the Tomcat manager webapp installed on each instance, so we can play with the webapps, and see how many active sessions there are.

3. Configure the ports and/or addresses for each instance

Tomcat listens to at least three network ports, one for the shutdown command, one for accepting requests on HTTP and one for accepting connection on AJP port. Two instances of Tomcat can’t listen to the same port number on the same IP address, so we will edit our server.xml files to change the ports they listen to.

The first port to look at is the shutdown port. This is used by the command line shutdown script (actually, but the Java code it runs) to tell the Tomcat instance to shut itself down. This port is defined at the top of the server.xml file for the instance.

1        

Make sure each instance uses a different port value. The port value will normally need to be higher than 1024, and shouldn’t conflict with any other network service running on the same system. The shutdown string is the value that is sent to shut the server down. Note that Tomcat won’t accept shutdown commands that come from other machines.

Unlike the other ports Tomcat listens to, the shutdown port can’t be configured to listen to its port on a different IP address. It always listens on 127.0.0.1.

The other ports Tomcat listens to are configured with the elements, for instance the HTTP or JK listeners. The port attribute configures which port to listen to. Setting this to a different value on the different Tomcat instances on a machine will avoid conflict.

Of course, we’ll need to configure whatever connects to that Connector to use the different port. If a web server is used as the front end using mod_jk, mod_proxy, or the like, then this is simple enough – change our web server’s configuration.

In some cases we may not want to do this, for instance you may not want to use a port other than 8080 for HTTP connectors. If you want all of your Tomcat intances to use the same port number, you’ll need to use different IP addresses. The server system must be configured with multiple IP addresses and the address attribute of the element for each Tomcat instance will be set to the appropriate IP address.

Ports to be changed listed below:

    8005
    8080
    8009
    8443 

4. Startup

Startup scripts are a whole other topic, but here’s the brief rundown. The main different from running a single Tomcat instance is we need to set CATALINA_BASE to the directory you set up for the particular instance you want to start (or stop).

Put the following in users .bash_profile or .bashrc:

1.       JAVA_OPTS="-Xmx800m -Xms800m"
2.       CATALINA_HOME=/opt/tomcat
3.       CATALINA_BASE=~/tomcat
4.       export JAVA_HOME JAVA_OPTS CATALINA_HOME CATALINA_BASE

Save and close the file. Start the Tomcat with following command

    $CATALINA_HOME/bin/catalina.sh start

That’s it now we can run multiple instances of Tomcat without making the redundant copies of Tomcat binaries and libraries.

Explaining Server Load

Introduction:
To explain what server loads really are and how to control them, how to watch loads and signs of server trouble.

Server Load Explanation
The load average tries to measure the number of active processes at any time. As a measure of CPU utilization, the load average is simplistic, poorly defined, but far from useless. High load averages usually mean that the system is being used heavily and the response time is correspondingly slow. What’s high? Ideally, you’d like a load average under, say, 3, Ultimately, ‘high’ means high enough so that you don’t need uptime to tell you that the system is overloaded. When seeing the results of the load averages, they are for the past 1, 5, and 15 minutes.

Checking the servers load
There are a few different ways to keep an eye on your servers load, the first thing you need to do is login to your server by SSH.

Method 1 using the uptime command:
The uptime shell command produces the following output:

    [sandbox2:~]% uptime
    09:53:04 up 34 days, 14:40, 1 user, load average: 0.01, 0.03, 0.00

It shows the time since the system was last booted, the number of active user processes and something called the load average.

Method 2 using the w command:
The w command produces the following output:

    [sandbox2:~]% w
    09:52:14 up 34 days, 14:39, 1 user, load average: 0.02, 0.04, 0.01
    USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
    mir ttyp0 :0.0 Fri10pm 3days 0.09s 0.09s bash
    giles pts/0 192.9.200.50 9:40am 0.00s 0.29s 0.15s w

Notice that the first line of the output is identical to the output of the uptime command.

Method 3 using the top command preferred:
The top command is a more recent addition to the UNIX command set that ranks processes according to the amount of CPU time they consume. It produces the following output:

    top – 09:54:47 up 34 days, 14:42, 1 user, load average: 0.07, 0.05, 0.01
    Tasks: 371 total, 1 running, 370 sleeping, 0 stopped, 0 zombie
    Cpu(s): 0.1%us, 0.1%sy, 0.0%ni, 99.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
    Mem: 37043972k total, 24892516k used, 12151456k free, 284460k buffers
    Swap: 39092216k total, 0k used, 39092216k free, 3825204k cached

    PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
    31963 root 15 0 12868 1212 720 R 1.9 0.0 0:00.02 top
    1 root 15 0 10348 692 580 S 0.0 0.0 0:05.16 init
    2 root RT -5 0 0 0 S 0.0 0.0 0:10.52 migration/0
    3 root 34 19 0 0 0 S 0.0 0.0 0:00.04 ksoftirqd/0

We like to use the top command because it also shows server uptime, memory information and the list of processes that you can sort by CPU usage, etc.

So what is a good load, bad load and in between?
I know you’re thinking, so what is a good system load or what is a bad load? Anything around 1.0 and below is fine, try to stick to under 1.0 for regular load averages. If you notice your server slowing down, check the load first. We hosted a site that was mentioned on the media (TV, News, Radio) recently and the server load skyrocketed because of the huge wave of traffic. The load went from 0.25 to 37.00 just because the server was getting hammered.

When your regular average starts to creep up around 2.0 then your server is very busy and you should consider getting another machine or upgrading your hardware. When I say regular average, I mean when the system is idle during the day and isn’t processing all your logs or backing up data.

Having an overloaded server can lead to many problems and should always be avoided. I hope this guide was helpful by giving you some more insight to server loads, what to use to monitor them and what is a good and bad load average.

Hope this was help full...

Linux Screen Shortcuts

Screen is a much under utilized program, which provides the following functionality:

    Remote terminal session management (detaching or sharing terminal sessions)
    unlimited windows (unlike the hard-coded number of Linux virtual consoles)
    Scroll back buffer (not limited to video memory like Linux virtual consoles)
    copy/paste between windows
    notification of either activity or inactivity in a window
    split terminal (horizontally and vertically) into multiple regions
    locking other users out of terminal

Key    Action    Notes
Ctrl+a c    new window   
Ctrl+a n    next window   
Ctrl+a p    previous window   
Ctrl+a "    select window from list   
Ctrl+a Ctrl+a    previous window viewed   
   
Ctrl+a S    split terminal horizontally into regions    Ctrl+a c to create new window there
Ctrl+a |    split terminal vertically into regions    Requires debian/ubuntu patched screen 4.0
Ctrl+a :resize    resize region   
Ctrl+a :fit    fit screen size to new terminal size    Ctrl+a F is the same. Do after resizing xterm
Ctrl+a :remove    remove region    Ctrl+a X is the same
Ctrl+a tab    Move to next region   
   
Ctrl+a d    detach screen from terminal    Start screen with -r option to reattach
Ctrl+a A    set window title   
Ctrl+a x    lock session    Enter user password to unlock
Ctrl+a [    enter scrollback/copy mode    Enter to start and end copy region. Ctrl+a ] to leave this mode
Ctrl+a ]    paste buffer    Supports pasting between windows
Ctrl+a >    write paste buffer to file    useful for copying between screens
Ctrl+a <    read paste buffer from file    useful for pasting between screens
   
Ctrl+a ?    show key bindings/command names    Note unbound commands only in man page
Ctrl+a :    goto screen command prompt    up shows last command entered

Hope this is helpful.

How to test Postfix SMTP using Telnet

1. Telnet to postfix mail server using TCP port 25.
Command is telnet (servername/serverip) 25
response should be as follows

    220 ********************************************

2. Turn on local echo on your telnet client so that you can see what you are typing.
On Win 9x and NT 3.5/4.0 Telnet client this done by selecting the “preferences” from the “terminal” pull down menu, and checking the local echo radio button.  For Windows 2000 telnet client, issue command “set local_echo”, from the telnet command prompt.

3. Issue the following smtp command sequence

ehlo (your domain name)(enter)

response should be as follows

    250-servername
    250-PIPELINING
    250-SIZE 10240000
    250-VRFY
    250-ETRN
    250-ENHANCEDSTATUSCODES
    250-8BITMIME
    250 DSN

mail from: (your Email Address)(enter)
response should be as follows

    250 2.1.0 Ok

rcpt to: (recipient address)(enter)
response should be as follows

    250 2.1.5 Ok

data(enter)
response should be as follows

    354 End data with (CR)(LF).(CR)(LF)

To: (recipient’s display name)(enter)
Subject: (Subject field of Email message)(enter)
(Enter you body text)(enter)
(enter)
(enter)
response should be as follows

    250 2.0.0 Ok: queued as queueid

quit(enter)
response should be as follows

    221 2.0.0 Bye
    Connection closed by foreign host.

I hope this save’s someone time.

Tuesday, March 11, 2014

Safely Deleting SQL Server ErrorLog File

Today I came across an issue on one of my windows server disk space was getting full rapadely.
Then I analyzed my servers disk space by tool called Free Disk Analyzer. I found that my SQL Servers Log folder was crossing the limit and ERRORLOG was growing in GB's.

You can download this Tool from here as well. Free Disk Analyzer .

Then first question came in my mind, Can I delete this directly?
Then Second question was is it Safe to do that?

By default SQL Server stores seven ERRORLOG files named as below

ERRORLOG
ERRORLOG.1
ERRORLOG.2
ERRORLOG.3
ERRORLOG.4
ERRORLOG.5
ERRORLOG.6

In my SQL Server 2008 R2, ERRORLOG files are located in the "C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\Log" folder. The ERRORLOG file contains the latest information and the ERRORLOG.6 file contains the older information.

My ERRORLOG files has grown too large in size, later I found a Stored Procedure to cycled the ERRORLOG file manually from the SQL server New Query window.

sp_cycle_errorlog

The data in the ERRORLOG files will be overwritten to ERRORLOG.1 file, you can make a copy of this old ERRORLOG.1 files to some external media if they are required for future reference.

It is now safe to delete the big size files.

Hope this helps...

Friday, March 7, 2014

Forgot the root password of your linux box

So you forgot your root password of your linux box?, well its not the end of the world if you do have physical / KVM over IP access to your box, you can start it on single user mode and reset it. Don’t know how?… then read on.

In all honesty, this fix really depends on which boot loader your box is running. Normally on modern Linux distros the loader will be GRUB.

In this case, how to get into single user mode using Grub boot Loader?

Restart your box, and when the GRUB splash screen appears follow these steps:

    Select/highlight the kernel you wish to boot using the up/down arrow keys.
    Press the e key to edit the entry.
    Select/highlight the line starting with the word kernel.
    Press the e key to edit the line.
    Append the letter S to the end of the line.
    Press ENTER to accept the changes.
    Press the b key to boot the kernel with the modified command line.

This will take you into single user mode and should provide a bash shell to execute commands.

From this point on you probably can use the passwd command to change the root password.

In you are using the LILO boot loader, when you see the LILO screen press CTRL+X, you will see boot propmt

    boot:

Here after boot: type

    linux single

This will boot the system in single user mode. When you reach the # prompt, type:

    passwd root

Once you reset the password you can then type init 3 or init 5. init 3 will start multi user boot without X and init 5 will start multi user mode with X/

If LILO is not set to wait at the boot menu, you can halt the boot process by pressing any key in the split second before lilo tries to boot the kernel.

By the way, in case you want to change the lilo timeout value, you can do it from:

/etc/lilo.conf  ( 0 means no wait )

once changed type:

    lilo -v

Apache DocumentRoot must be directory

You have created a separate partition for you htdocs folder and mounted it on /path/to/documentroot. However, when you try to start Apache you get following error ‘DocumentRoot must be directory’.

This is due to SELinux to get Apache working you have 2 ways

Crude way:- disable SELinux altogether
Eligible way:- set SELinux context to that directory

___________________________________________________________
Crude way:- disable selinux altogether

Follow below steps:

$ vi /etc/selinux/config
$ Change 'SELINUX=enforcing' to 'SELINUX=disabled'
$ Restart your system
____________________________________________________________   
Eligible way:- set SELinux context to that directory

First check the SELinux context of the directory with following command.

$ ls -laZ /path/to/documentroot

Follow below steps:

$ chcon -R -h -t httpd_sys_content_t /path/to/documentroot

Verify the SELinux context

$ ls -laZ /path/to/documentroot

Restart your Apache and it works :).

Hope this helps and saves your time.

Thursday, March 6, 2014

How to Migrate putty sessions from one machine to other.

Exporting Your PuTTy Configuration

Putty stores its settings in the Windows registry. To save a backup of your Putty settings, you'll need to export this registry key to a file. 

HKEY_CURRENT_USER\Software\SimonTatham

(Simon Tatham is the original developer responsible for PuTTy)

1. Click Start->Run and type "RegEdt32" in the "Open" dialog. Click "Ok"

2. One RegEdt32 starts, you'll be presented with an application which looks something like:

3. Press "Ctrl+F" to bring up the Find dialog. Enter the name of the key, "SimonTratham" in the "Find What" field, and make sure only "Keys" is checked in the "Look At" section of the dialog. Finally, click "Find Next"


4. The search may take a while, reminding us that the Windows Registry is a large and mysterious place where dragons be. Let's use these few seconds to reflect on the fact that you should never, ever, never change things in the registry unless you are absolutely, positively, totally, completely, 100%dead sure that you know exactly what you're doing. When the search completes we'll see the key name for which we're looking.

5. Click File->Export. Give your file an appropriate name like, "putty.reg" and click "Save"

6. We're done! Save the putty.reg file somewhere safe. The file doesn't contain any passwords or actual SSH key values so, it's relatively safe from prying eyes. Still, it does contain your configuration and that kind of data is a private matter.

Importing Your PuTTy Configuration

To import your saved PuTTy configuration on any other Windows computer simply copy your exported registry key, right click on the file and click "Merge"

Windows will ask you for confirmation that you want to import this set of registry values. We know this file is safe, because we created it but, you should never import registry information from an unknown source.

That's all you need to know about moving your PuTTy configuration from one machine to another. This can be really useful information when upgrading to a new PC or, if you're an office IT guy where your users all have a standard list of servers they need to connect via SSH, you can create a reference configuration on once machine and "share" it between every computer in the office. 

4. The search may take a while, reminding us that the Windows Registry is a large and mysterious place where dragons be. Let's use these few seconds to reflect on the fact that you should never, ever, never change things in the registry unless you are absolutely, positively, totally, completely, 100%dead sure that you know exactly what you're doing. When the search completes we'll see the key name for which we're looking.

5. Click File->Export. Give your file an appropriate name like, "putty.reg" and click "Save"

6. We're done! Save the putty.reg file somewhere safe. The file doesn't contain any passwords or actual SSH key values so, it's relatively safe from prying eyes. Still, it does contain your configuration and that kind of data is a private matter.

Importing Your PuTTy Configuration

To import your saved PuTTy configuration on any other Windows computer simply copy your exported registry key, right click on the file and click "Merge"

Windows will ask you for confirmation that you want to import this set of registry values. We know this file is safe, because we created it but, you should never import registry information from an unknown source.

Windows will ask you for confirmation that you want to import this set of registry values. We know this file is safe, because we created it but, you should never import registry information from an unknown source.

That's all you need to know about moving your PuTTy configuration from one machine to another. This can be really useful information when upgrading to a new PC or, if you're an office IT guy where your users all have a standard list of servers they need to connect via SSH, you can create a reference configuration on once machine and "share" it between every computer in the office. 

VMware CPU and Memory Reservations: Fixing Insufficient resources to satisfy configured failover level for HA


This post comes from a few days of poring over manuals as well as some technical support. This is a good one. The error came from trying to power on a VM in our VMware cluster and we would get these errors:
“Insufficient resources to satisfy configured failover level for HA”


And this alert on our cluster
“Insufficient resources to satisfy HA failover level on cluster vmCluster in vmTST”
Our way of thinking was we had to power one off to power another one on.
But that didn’t work.
Here we is the actual solution. (p.s. Great VMware HA education for me on this one!)
PROBLEM SOURCE: VMware HA is turned on and you are violating constraints
VMware HA is turned on, and you have it configured so that there is a certain amount of resource reserve for failover. By turning on this VM, you are going to dip into that resource reserve and so VMware is telling you “Nope, not turning it on….”
There is a quick fix to get the VM turned on (one good way, one bad way), and then there are two long term fixes for you to consider. In my case, the first one was faster, while the second one was better for my environment.
My VMware environment
Datacenter: vmTST
Cluster: vmCluster
OS: ESXi 4.1.0
Five (5) servers in a cluster.
My VMware Cluster Errors
As mentioned above:
“Insufficient resources to satisfy configured failover level for HA”
and
“Insufficient resources to satisfy HA failover level on cluster vmCluster in vmTST”
TWO WAYS TO DO QUICK FIX
1.Turning off HA (popular, and I would say WRONG)
2.Disable Admission Control (much better!!)
#1: Turning off HA (though I recommend against)
This is the solution I saw on some forums (including vmware forum). After looking at it more, I recommend against it and I’ll explain why, but here it is:
VSphere Client: Browse Inventory -> Hosts and Clusters
Edit VMware cluster settings
Right Click on Cluster name -> Edit Settings
Turning off HA
While this works, if you do this, whenever you turn it back on, it has to do a recalculation for the HA failover. Bad, especially for testing or doing temporary power ons.
#2: Disable “Admission Control” (better IMO)
Better to disable “Admission Control” so VMS will power on despite violating availability constraints. This way your HA is still on. In the long run, though, it is better to fix your issue.
Same window, but next bullet item on the left:
LONG TERM FIX: TWO WAYS
There are two things I ended up having to look at. One was pretty good long term fix and that I had found suggested on forums including VMware forums.
The second is the actual fix to my problem, the best one in the long term
FIX #1: Change from “Host Failures Cluster Tolerates” to “Percentage of cluster resources reserved as failover spare capacity”
In other words, instead of telling VMware you want to have enough resource reserve so that you can lose one host, you are telling VMware you want to have a certain percentage of resources unused for failover.
We had it configured to lose one host. So by switching to a percentage it was a quick and easy fix for my environment.
VMware HA: Host failures cluster tolerates (?)
So if we look at the “VMware HA” window, you’ll see that my “Host failures cluster tolerates” was set to 1. Now with 5 servers you would think that means “20%” but that’s not so. Because what if one of your VMs (or more) for whatever reason took up 75% of your resources, then by worst case calculation you could only have one VM on your five node cluster.
A worst case calculation of your largest VM will determine what’s called a “slot” size. The VMware HA will then calculate how many total “slots” can be used which determines how many total VMs you can have powered on.
When this option is chosen, from what I’ve read on VMware forums, the calculations are VERY conservative.
Find Your Slot Size: VMware Cluster Summary -> Advanced Runtime Info
VMware Advanced Runtime Info: Slot sizes
So you can see above, worst case scenario, one slot size is 2507Mhz, 4256 MB. With that in mind, there are 55 slots available on my five node cluster. There are a total of 156 VMs out of 55.
This means I would have to power off 102 VMs to get to 54 powered on VMs leaving one slot open to power the new one on… (YIKES!0
Changing To Percentage: First Check Resource Usage
Out of curiosity, I checked the actual resource usage in my cluster
If you tally up all the green bars in CPU, I could fit all the CPU usage of every VM on one host.
If you tally up all the green bars in Memory, I could fit all the memory usage in about three hosts.
So why can’t I power on a VM? Because the calculation is *THAT CONSERVATIVE* for the “Host failure cluster tolerates” option
VMware HA: Switch to percentage
Now, the first time I did this, I chose “20%” which prorated to one server out of the five being free.
And I was able to power on a VM
On a whim, I kept upping the percentage and I got as high as 75% before I decided to stop, thinking I was doing something wrong.
Part of it was that the VM I was powering on was very very small in resource usage (and later I found out also it had 0 reserve configured with it) which is probably why it powered on even at 75% failover spare capacity.
Anyhow, so in a pinch, this is one way to configure some amount of reserve AND be able to power on your VMs, at least if your resource usage somewhat mirrors mine (see previous picture)
FIX #2: Best Long Term Fix: Determine WHY the cluster resource reserve is so high and see if it is actually needed, or if it is just poorly configured
In the end this was the actual fix for us, because it delved into the actual source of the problem. Which was to find out:
WHY the heck was our VM slot size so BIG?
Because obviously all five hosts combined were using VERY LITTLE CPU and RAM. Less than 20% on CPU (it could fit all on one server), and less than 50% on RAM (it could fit on two to three servers).
It turns out: The slot size is not based on usage, it is based on a VM resource reservation.
So here is how to check your resource reservation for your Vms.
VMware Cluster: Resource Allocation for CPU and Memory
CPU
(The dashed lines are my VM names which I blanked out)
Click on the “CPU” button and look for the “Reservation” column and sort by largest to smallest.
Memory
(The dashed lines are my VM names which I blanked out)
Click on the “Memory” button and look for the “Reservation” column and sort by largest to smallest.
As you can see, there are many VMs with resource reservation. This means as soon as the VM is powered on, it will reserve this much resource REGARDLESS IF IT IS NEEDED OR NOT!
But as you can see by actual usage, we are not even near to capacity, there is no real reason for us to reserve that much.
One of the culprits: it turns out many of our templates we use to clone/deploy VMs had resources reservation already set, so each time we made a new VM it had a resource reservation.
VMware Cluster: Virtual Machines Actual Usage
Go to the tab “Virtual Machines” now and you can see actual usage. There is a column “HOST CPU – Mhz” and “Guest Mem – %”. These show actual usage by the VM.
I sorted alphabetically here and referenced the previous two pictures (VMs with the highest reservations) and then checked this list to see actual usage. Sure enough, many of our VMs were not using that much resource (as you can tell from earlier graphs)
Next step: contact VM owners to see if the VM was in typical usage. If so, get permission to turn the resource reserve down or even off.
VMware: Right Click -> Edit Settings
To configure resource reserve, right click on the VM and Edit Settings
VMware: CPU reservation and Memory reservation
Here I turned the CPU Resource reservation and memory reservation low or to zero
REMEMBER TO CONSULT YOUR USER FIRST TO SEE IF VM IS IN TYPICAL USE
VMware HA: Advanced Runtime Info Results
Now go back to your Advanced Runtime Info Results… (you might have to turn the VMware HA to “Host failover cluster tolerates” if you had changed it to the percentage as an intermediate fix)
When all was said and done, I went from 55 slots to 550 slots.
And from being in the “red” of 101 VMs I’d need to power off to power one on to being in the “green” of 394 VM slots available.
CPU slot size went down a factor of 10
Memory slot size went down a factor of 20
NICE!!!
Hope this has been helpful!