I have written a number of articles regarding i5/OS and OS/400 performance in the past in which I have discussed memory management using the built-in work management capabilities of the operating system. I have also covered the use of functions such as disk balancing to ensure that your data is evenly spread across disk arms. However, something I have found that many customers are not getting their head around is LPAR performance.

Every 5xx System i out there – and System p, for that matter – is an LPAR system even if there is only one operating system instance running on it. The POWER Hypervisor is resident on all POWER5 and POWER6 systems and virtualises the hardware on the box. You may have a small i520 which replaced an older 170, for example, running just one copy of i5/OS. With a single partition system, the POWER Hypervisor is told that the partition owns all the hardware in the system. This means that, with the aid of the HMC, we can turn a single LPAR machine into a multi-LPAR machine simply by reassigning hardware.

Allocation issues

Machines that are running multiple LPARs require some careful consideration in terms of the allocation of CPU and memory resources. For most of this article I am going to use a 500 CPW i520 as an example. This machine has a single 1.5GHz POWER5 processor enabled, which is capable of providing 2,500 CPW; IBM ‘slugs’ this processor so that it provides you with around 22% of its processing capability. Compare this to the new i515 and i525s which use the 1.9GHz version of same processor but have the restrictions removed. We are also going to assume that we will be running two LPARs on this system. One LPAR, naturally, will be i5/OS, the second partition could be i5/OS, Linux or AIX.

Using the HMC allows me to look at the properties of the i520 and determine what processing resource is installed and available to me. Figure 1 shows that my machine has one processor installed, but only 0.22 configurable to my LPARS. Basically, this is the IBM slugging. For those of you who have used the HMC, Figure 1 may look a little different. I am using HMC version 7.3, otherwise known as eClipz HMC. The eClipz stands for Enhanced Core Logic for i,p and z. The HMC at 7.3 is now a browser-based application for both local and remote use.

Figure 1

What does this mean? When I create LPARs on this system I cannot assign any more than 0.22 processors in total. This seems ridiculous, as in the physical world there is no such thing as 0.22 of a processor; we either have a processor or we don't. In a virtual world we can assign a partition to be as small as 0.10 of a physical processor. This means that we can run two LPARs on this system. You can see that in Figure 1 we have an i5/OS LPAR with 0.12 of a processor and a Linux partition with 0.10 of a processor.

Lack of flexibility

I will be perfectly honest and say that this is not really very flexible and can be pretty slow. If you do the sums, 0.12 of a processor that is capable of delivering 2,500 CPW gives my i5/OS LPAR about 300 CPW. Not enough to run WebSphere or Java applications, but OK for standard 5250 workloads. Let’s remember that CPW is calculated based upon a balanced system consisting of CPU, memory and disk, so I am being very general saying that we get 300 CPW from 0.12 of a processor.

POWER5 processors provide a function called Symmetrical Multi-Processing (SMP). This allows an operating system running on POWER5 to submit two tasks to the processor at any one time. In effect, the operating system thinks that each physical processor is two processors. In i5/OS we can control the use of this by setting the QPRCMLTTSK system value. This is set to two by default and there is normally no need to change it.

When we create a partition profile in the HMC we need to set the number of virtual processors the operating system will ‘see’. By default this value is set to ten times the number of physical processors assigned. So, if I create an LPAR and specify that I want it to use two physical processors, the virtual processor value will default to 20. In effect, we are telling that LPAR that it can submit 40 tasks concurrently to two physical processors. This could lead to very poor performance.

By way of example

Figure 2 shows the processor allocation properties for an i5/OS partition. On our i520 the maximum virtual processors is set to two. This is because the system has rounded up to the next whole number. Do I really want to tell i5/OS that it can submit two tasks simultaneously to this processor? As with all things performance-related, it depends. In some instances this may work for you, in other cases it may be better to set maximum virtual processors to one. More than anything, be aware of the virtual processor settings for your LPARs. I normally set these to two times the number of physical processors assigned to the LPAR; it seems to be a good starting point.

Figure 2

One of the features of the POWER5 processors is uncapped processing. Let's say that you have three processors available for use and you set your LPAR to use a maximum of two processing units and three virtual processors. A capped LPAR will never use more than the maximum number of processors configured. There may be times when your LPAR is running flat out at 100% CPU utilisation while your other LPARs are idle. This means that we are only using two out of our three processors, which seems like a waste.

Uncapped LPARs allow the POWER Hypervisor to temporarily assign more processing resource to a busy LPAR. So, if our production partition is running at 100% CPU, but the remaining partitions are idle, then the system will allow our production LPAR to use three processors rather than two. The maximum amount of uncapped processing resource allowed is dependent on the value for maximum virtual processors. When this happens you will see the CPU utilisation value in i5/OS exceed 100%. If our partition is running at 100% with a maximum of two processing units and we then utilise a third processor we could see the CPU utilisation rise from 100% to 150%, because we've utilised the third processor. If your system is consistently running at more than 100%, you should consider how your LPAR’s processing resources are configured.

I still see customers worrying about using uncapped processing, but it works well. Remember that the POWER Hypervisor treats each LPAR as a job. What we define in the HMC is how much CPU and memory we allocate. Only one LPAR can be ‘active’ in a processor pool at any one time; when its allocation of time is up the next LPAR will be loaded into the CPU. If an uncapped processor is using spare processor resource and another LPAR needs CPU, it will get the CPU.

LPAR timeslices

We always talk about an LPAR having parts of a processor. If we have an LPAR with 0.12 processing units assigned, what we really mean is the LPAR will be allocated 1.2 milliseconds of time in the processor. It will then be removed from the processor(s) and the next LPAR will be loaded in. So when we talk about using one processor, we really mean 10 milliseconds of time in the processor. If we have three physical processors available to our LPARs then we have 3 x 10 milliseconds of processing time available. So an LPAR using 2.5 processing units will be allocated 25 milliseconds of the 30 milliseconds available.

When you consider LPARs as jobs in the POWER Hypervisor it makes understanding the performance of an LPAR easier. We allocate a timeslice for each LPAR to use in the processor pool which it cannot exceed by default. However, if we set an LPAR to be uncapped then it can exceed its timeslice, but if another LPAR needs CPU, it will get it. The POWER Hypervisor will always make sure an LPAR gets its minimum or desired processing units (timeslice).

Adding some intelligence – APRM

So, you've taken the brave step of using uncapped LPARs on your system and all is working well. One evening you find that your production LPAR appears to have slowed down and isn't using the additional CPU that it has been using before. You are only getting 100% utilisation, rather than the 150% you are expecting for part of your overnight processing. Eventually you discover that another LPAR is running an ad-hoc job which means that particular LPAR is also running at the same time as your production overnight batch. How can you prevent this? You could set the second LPAR to capped, but that is not ideal as there are times when you want the second LPAR to utilise unused processing resources. The POWER Hypervisor and HMC doesn't provide any kind of intelligent control of resources, nor does it provide an audit trail of the CPU and memory resources used by the LPARs.

IBM recently published a Redbook on the Automatic Partition Resource Manager (APRM). This is a third-party product that allows you to do a number of things with your i5/OS and Linux LPARs. You can set up business rules to determine what and when resources are available to the LPARs, including 5250 interactive capabilities.

In the example above we would specify that our second LPAR cannot use unused processor resource during 18:00 and 22:00, for example, to make sure that it cannot hog resources while our production LPAR is running its overnight jobs. We can also prioritise the partitions to ensure that our production LPARs have a higher priority than our DR/Dev LPARs.

What APRM is doing here is extending the flexibility of the LPARs in terms of using CPU, memory and 5250 resources. By applying business-type rules we can have much greater control over when and where resources are allocated between LPARs, thus avoiding other LPARs potentially taking resources, when more important workloads require the resources. APRM also has auditing and reporting capabilities, so you can see what resources were allocated and when.

Another great feature of APRM, which is not associated with LPAR performance, is the ability to execute CL commands to move resources between LPARs; for example, moving a shared tape IOA and IOP between LPARs by running the ACQAPRMDEV command. The fact that IBM’s System i team have written a Redbook about this product means that they are interested in how it can supplement the POWER Hypervisor features.

Hopefully, this article has helped you understand a little more how LPARs are viewed by the POWER Hypervisor and how you can tune your LPARs to get the best performance. The APRM tool is a great add-on for your LPARs to really customise how you want your system's resources allocated to the active partitions. Even though this is a chargeable item, I think it is well worth investing in if you are running multiple production partitions on your iSeries or System i.