Hands-On 7: Increase Compute pod’s CPU limit

Step-by-step instructions
What have we learned ?
References

In this hands-on, we will first look at the default CPU and memory limits and request for the SAS Compute server pods, and also monitor the CPU and memory consumption of the SAS Compute Server pod while a SAS program is running. Then we will change the max and default values for the CPU limits and memory and run the program again to see if the processing times are changing. (Tip #3)

Step-by-step instructions

Check the default CPU and Memory limits of your SAS Compute session

Open Google Chrome and click the SAS Studio bookmark or if you use the SAS App, click the hamburger icon (top left) and select Develop Code and Flows.
Login as student/Metadata0.
Choose Yes when asked to opt in to all of your assumable groups.
Click Program in SAS.
Switch from the SAS Studio Compute Context_SHARED Context to the standard SAS Studio Compute Context, confirm by clicking on Change.
Wait for the Studio Compute context to be ready, then check the current CPU and Memory requests and limits for the Compute container of the pod that you have just started.
- Run the following command in the Root Terminal window:
```
kubectl -n edu get pod -l 'launcher.sas.com/username=student' -o json | jq -r '.items[0].spec.containers[] | select (.name=="sas-programming-environment") | .name, .resources '
```
- You should now see the default CPU and memory limits of the pod: 2 cores and 2Gi of memory.
- Now let’s put a watch on the CPU and memory utilization for the pod, by running this command in the Root Terminal window:
```
watch 'kubectl -n edu top pod -l launcher.sas.com/username=student'
```
- With this command you can see the real time consumption of CPU and memory of your SAS Compute session.

Run a sample SAS Program

Come back to your SAS App (or Google Chrome) and the SAS Studio programming interface.
Click Program in SAS.

Copy and paste the following SAS Code that creates a sample dataset and perform a “high-performance” procedure named HPGENSELECT:

options fullstimer; /*display resource usage in the log*/
/*generate a sample dataset*/
%let numCopies=400000;
data getStarted;
input C1-C5 Y Total;
do i=1 to &numCopies;
output;
end;
drop i;
datalines;
3 1 1 3 2 28.361
3 0 3 1 2 39.831
3 2 2 2 1 17.133
2 0 0 3 2 12.769
2 1 0 1 1 29.464
2 1 0 2 1 4.152
2 1 0 1 0 0.000
2 1 1 2 1 20.199
2 0 0 1 0 0.000
1 1 3 3 2 53.376
2 2 2 1 1 31.923
3 2 0 3 2 37.987
2 2 0 0 1 1.082
2 0 2 0 1 6.323
3 0 0 0 0 0.000
2 1 2 3 2 4.217
1 2 3 1 1 26.084
1 0 0 1 0 0.000
3 2 2 2 0 0.000
1 3 1 1 2 52.640
3 0 1 2 1 3.257
0 2 3 0 5 88.066
2 2 1 0 1 15.196
1 3 1 0 1 11.955
1 3 1 2 3 91.790
1 1 2 3 7 232.417
1 1 1 0 1 2.124
1 0 0 0 2 32.762
1 2 3 0 1 25.415
2 0 1 2 1 42.753
3 2 2 3 1 23.854
0 0 2 3 2 49.438
0 0 2 3 4 105.449
0 2 3 0 6 101.536
3 1 0 0 0 0.000
0 1 0 1 1 5.937
0 0 0 3 2 53.952
0 1 0 3 2 23.686
1 3 1 1 1 0.287
1 3 0 3 7 281.551
3 2 1 1 0 0.000
1 0 0 1 0 0.000
0 1 1 2 3 93.009
1 0 1 0 2 25.055
2 2 2 3 1 1.691
3 2 3 1 1 10.719
3 0 3 3 1 19.279
0 0 2 1 2 40.802
2 3 0 3 3 72.924
2 0 3 0 1 10.216
0 1 2 2 2 87.773
1 2 3 1 0 0.000
2 0 3 1 0 0.000
0 3 0 0 2 62.016
3 2 2 1 3 36.355
3 2 0 3 1 23.190
0 1 2 1 1 11.784
1 2 2 2 5 204.527
0 1 1 2 5 115.937
1 1 3 2 1 44.028
2 1 3 1 4 52.247
1 0 0 1 1 17.621
3 1 2 1 2 10.706
2 0 2 3 3 81.506
1 0 0 2 2 81.835
1 2 0 1 2 20.647
2 2 2 0 1 3.110
2 3 0 0 1 13.679
2 2 3 2 1 6.486
3 2 2 1 2 30.025
0 3 1 3 6 202.172
2 3 1 2 3 44.221
3 0 0 0 1 27.645
3 3 0 3 2 22.470
3 2 0 2 0 0.000
3 0 2 0 1 1.628
3 1 0 2 0 0.000
2 3 3 0 1 20.684
1 0 2 0 4 108.000
1 2 2 1 1 4.615
2 3 2 2 1 12.461
3 2 0 1 3 53.798
1 1 2 0 1 36.320
0 3 0 0 0 0.000
0 3 2 0 1 19.902
2 3 1 0 0 0.000
2 2 1 3 2 31.815
3 3 0 0 0 0.000
2 1 3 3 2 17.915
2 3 2 3 2 69.315
3 1 2 1 0 0.000
0 1 1 1 4 94.050
1 1 1 3 6 242.266
2 0 3 2 1 40.885
0 1 1 2 2 74.708
2 2 2 3 2 50.734
0 2 2 1 3 35.950
3 3 1 1 1 2.777
1 2 1 3 5 118.065
3 2 1 2 0 0.000
;
proc hpgenselect data=getStarted;
class C1-C5;
model Y = C1-C5 / Distribution=Poisson Link=Log;
run;

Run the SAS program (Hit the F3 key or click the button).
While your program runs, bring up the Terminal window where you have set the watch on the CPU and memory usage, you should see the CPU and memory used value change.
At the end, click the Log tab and take a note of the “real time” and “user cpu time” for the last operation (PROC HPGENSELECT).
In the Root Terminal window, you can now interrupt the “watch” command by typing <Ctrl+C>.

Change the default CPU and Memory limits for the Compute Server

There are two configuration tasks to perform:

Increase the maximum possible CPU and RAM limits for SAS Compute Server pod.
Inrease default CPU and RAM limit for the SAS Compute Server pod.

--- Expand this ONLY if you want to use the SAS Environment Manager UI to do it ---

Switch to Environment Manager (select Manage Environment) and open the configuration page .
Using the drop-down list at the top, choose All services.
Select Launcher service (You can type “Launcher” in the filter to quickly locate the service).
The instances are expanded by default. Collapse all the items by clicking on the icon .
Now, beside the sas.launcher.max, click the Edit icon to modify the maximum values for CPU requests, CPU limits, memory requests, and memory limits.
- Set the CPU limit to 4 (instead of 2 by default).
- Set the Memory limit to 16Gi (instead of 2Gi by default).
Click Save.
Now, for the sas.launcher.default instance, click the Edit icon beside to modify the default values for the CPU and memory limit.
If we increase the CPU limit, we should also increase the memory limit so each CPU have more memory to work with.
- Set the cpu.limit value to 4.
- Set the Memory limit to 8Gi (instead of 2Gi).
Click Save.

Otherwhise follow the instructions below to do it from the command line (faster):

Increase the max limits for CPU and memory

If we increase the CPU limit, we should also increase the memory limit so each CPU have more memory to work with.

Get the identifier for sas.launcher.max definition:

ITEM="sas.launcher.max"
MEDIATYPE=$(sas-viya configuration configurations download -d $ITEM | jq -r '.items[]["metadata"]["mediaType"] ')

Create a JSON file (increase maximum “cpu.limit” from 2 to 4 and increase maximum “memory.limit” from 2Gi to 16Gi):

tee ./update-$ITEM.json > /dev/null << EOF
{
    "items": [
        {
            "metadata": {
                "isDefault": false,
                "mediaType": "${MEDIATYPE}"
            },
            "cpu.limit": "4",
            "cpu.request": "100m",
            "memory.limit": "16Gi",
            "memory.request": "500M"
        }
    ]
}
EOF

Apply the change:

sas-viya configuration configurations update --file ./update-$ITEM.json

With result similar to:

"PATCH" "/configuration/configurations" complete from "./update-sas.launcher.max.json".

(Optional) Validate the change in SAS Environment Manager.

Navigate to Configuration (left-menu) > Definitions (View pull-down) > sas.launcher.max (from list) > Launcher service (parameter group) > cpu.limit (item) should have value of 4 (originally 2) and memory.limit (item) should have a value of 16Gi (originally 2Gi).

Increase the default CPU and memory limit

Get the identifier for sas.launcher.default definition:

ITEM="sas.launcher.default"
MEDIATYPE=$(sas-viya configuration configurations download -d $ITEM | jq -r '.items[]["metadata"]["mediaType"] ')

Create a JSON file (increase default “cpu.limit” from 2 to 4 cores and the memory from 2Gi to 8Gi):

tee ./update-$ITEM.json > /dev/null << EOF
{
    "items": [
        {
            "metadata": {
                "isDefault": false,
                "mediaType": "${MEDIATYPE}"
            },
            "cpu.limit": "4",
            "cpu.request": "100m",
            "memory.limit": "8Gi",
            "memory.request": "500M"
        }
    ]
}
EOF

You might have noticed that we also increase the memory limits. If we increase the CPU limit, we should also increase the memory limit so each CPU have more memory to work with.