by Deiby Gómez

You have few free space in the current diskgroup where your principal database is running, you have requested your manager more disks in order to increase the free space, your manager contacted Storage Team and finally you have the approval for adding 10 disks to your current diskgroup. Since all those 10 disks have to be added at the same time and since the database has around 1 TB you are worried because you believe the rebalance activity will take a lot of time impacting in business hours, you have to create a plan and decide either using a high value for asm_power_limit impacting the performance of your principal database or using a low value for asm_power_limit impacting (maybe) the business hours. So that you are thinking about how many hours the rebalance operation will take using low asm_power_limit value, you believe that using low value you will not impact the performance and also you will not impact the business hours because the rebalance activity will finish at an acceptable time, that is what we will try to find out, a way in order to have a clue regarding the duration of the rebalance operation which is given directly by the amount of MBs that has to be reallocated.

First you have to understand that there is no a magic view which will give you the right duration, of course it exists v$asm_operation.EST_MINUTES but you can take a look in many forums and articles and you will find that that column is not exact, even it could have a lot of difference between the correct time and the v$asm_operation.EST_MINUTES. And also we are trying to be proactive, I mean, how to find out the amount of MB without altering any disk.

You also have to understand that the rebalance operation time depends of many things, among them:

  • Asm_power_limit
  • Diskgroup redundancy
  • Disks throughput
  • Disks RAID
  • Disk provider
  • AU size
  • How many disks are being added or dropped
  • Failure groups


So I would write thousands of pages about it and perhaps we will have finally a magic formula but I am pretty sure that it will be too complex. So what I will show you is how to find a unity in order to measure the rebalance activity time. This unity is named “Allocation Unit” and I am pretty sure that you know what I am talking about.

Let’s take a look into our diskgroup:

SQL> select group_number, name, total_mb, free_mb, type from v$asm_diskgroup where name='DATA2';

GROUP_NUMBER NAME                  TOTAL_MB      FREE_MB TYPE
------------ ------------------------------ ---------- ---------- ------
       1 DATA2                 12284        11931 EXTERN

And also the disks information inside the diskgroup:

SQL> select name,total_mb from v$asm_disk where group_number=1;

NAME                 TOTAL_MB
------------------------------ ----------
DATA2_0000                12284

As you can see there is a 12GB disk and it has external redundancy.

Now let’s see how many extents does that disk have because a part of those extents will be reallocated across all the disks including new disks.

SQL> Select count(pxn_kffxp) Extents, disk_kffxp disk_number, group_kffxp group_number from x$kffxp where group_kffxp=1 group by disk_kffxp, group_kffxp order by group_kffxp,disk_kffxp;

       EXTENTS DISK_NUMBER GROUP_NUMBER
---------- ----------- ------------
       351         0          1

There are 351 Extents in that disk; some of those extents have to be reallocated across all disks including new disks.  

How many MBs do those Extents represent?

Every extent would have many Allocation Units. It depends of the redundancy of the diskgroup and how many failure groups you have. You can see the following field in order to see if the AU is mirror or primary. We also have virtual extent and physical extent but that is outside of the scope of this article.
 

x$kffxp.AU_KFFXP 0 for primary AU
1 for Mirror AU


Since I am using an External Redundancy Diskgroup and since I only have one failure group I only have one AU per Extent. That means:

SQL> select name, allocation_unit_size from v$asm_diskgroup where name='DATA2';

NAME                   ALLOCATION_UNIT_SIZE
------------------------------ --------------------
DATA2                        1048576

351 AU x (1 MB /1 AU )=351 MBs.
We have 351MB of data in that disk.

It makes sense because we saw the following information from v$asm_diskgroup:

SQL> select group_number, name, total_mb, free_mb, type from v$asm_diskgroup where name='DATA2';

GROUP_NUMBER NAME                  TOTAL_MB      FREE_MB TYPE
------------ ------------------------------ ---------- ---------- ------
       1 DATA2                 12284        11931 EXTERN

12284-11931≈353 MB

Conclusion:

We have 1 disk of 12GB (12284 MB) and we will add two disks of 12GB. That means that we will have 3 disks of 12GB. Since ASM redistributes all extents evenly across all disks it means that we should have 351/3 MBs in every disk.

Tips:

  • ASM always will redistribute data evenly across all disks on a percentage basis.
  • If you have disks with different size the data will be redistributed in the following way:
    • Every disk will have the same percentage of data, it means that if you have a 10G disk and other 100G disk you will have 10G of data in the second disk per every 1 GB in the first one.
  • The only reason that you have unbalanced disks is that rebalance activity was cancelled or the asm_power_limit=0.
  • If a rebalance activity failed then ASM recovers automatically that activity and start the rebalance activity again.
  • If we are talking about RAC configuration, then other ASM instance performs the recovering activity.


How many MB the rebalance activity will reallocate?

Formula: (TAU/N)*N-1
Where:
TAU= Total Allocation Unit to move.
N=number of disks

Note: this formula only can be used if every disk inside the diskgroup has the same size. Oracle recommends using disks with the same size.

(351AU/3)*2=234AU=234 MBs (we are using the default AU size)

So now you know how many MBs you have to move if you add 2 more disks. Once you know the MBs you can compute that number with Disk throughput in order to find out if you should use higher asm_power_limit or a lower value.

Reminder:

  • Each Extent is moved one by one if you are using asm_power_limit=1.
  • Each extent is locked, relocated and unlocked.


Well, so far we know that if we add another 2 diska in our diskgroup we will have to move 234 MBs. Do we have the exact value? Let’s see what happened if we perform the ADD DISK command:

SQL> Alter diskgroup data2 add disk '/dev/oracleasm/disks/ASMDISK4','/dev/oracleasm/disks/ASMDISK5';

Diskgroup altered.

Immediately I executed the following query:

SQL> Select operation, state, est_work, est_minutes from v$asm_operation;

OPERA STAT   EST_WORK EST_MINUTES
----- ---- ---------- -----------
REBAL RUN      232        0

As you can see 232 Extents had to be reallocated. We are very close of the correct number.

We have to do all this work if we want to be proactive, if you have to know how many MBs you have to move when you add or remove a disk without altering the diskgroup. As you can see this is a little tedious and it takes time, and also since we have humans we can do any mistake while we are playing with the numbers. In Oracle 12c we have a new sentence that perform all this work for us. The sentence “EXPLAIN WORK FOR” will compute the data in order to give you how many extents you have to move or reallocate without altering the diskgroup.

How it works:

Let’s use this diskgroup for the example

SQL> select group_number, name, total_mb, free_mb, type from v$asm_diskgroup where name='DATA';

GROUP_NUMBER NAME                  TOTAL_MB      FREE_MB TYPE
------------ ------------------------------ ---------- ---------- ------
       2 DATA                 15500         5863 EXTERN

You have to execute the new sentence EXPLAIN WORK FOR” this way:

SQL> explain work for alter diskgroup data add disk '/dev/oracleasm/disks/DISK2';

Explained.

The information about how many MB should be reallocated is in the view v$asm_estimate:



SQL> select * from v$asm_estimate;

GROUP_NUMBER STATEMENT_ID            TIMESTAMP    EST_WORK     CON_ID
------------ ------------------------------ --------- ---------- ----------
       2                    08-JUL-14        4259      0

So you can know the amount of the extents at an easier way. Is this value exact? Let’s see what happen if we alter the diskgroup:

SQL> alter diskgroup data add disk '/dev/oracleasm/disks/DISK2';

Diskgroup altered.

Immediately I executed the following query:

SQL> Select operation, state, est_work, est_minutes from v$asm_operation;

OPERA STAT   EST_WORK EST_MINUTES
----- ---- ---------- -----------
REBAL RUN     4259        0

As you know the value is exact. There are many new features in Oracle 12c however I like this since it saves a lot of work for me. Also for RAC it exists a new feature, which does a similar activity for many RAC commands. That is what we will talk about in my next article. Come here frequently and you will find more articles like this.
How to estimate the amount of MB for a rebalance activity in ASM 11g and ASM 12c.