Thursday, September 20, 2007

ORA-04043 in mount mode

There was a question at the OTN Database-General forum today about a problem when trying to describe the view dba_tablespaces.

The poster was getting an ORA-04043 error.


SQL> desc dba_tablespaces
ERROR:
ORA-04043: object dba_tablespaces does not exist

The first thing I thought about this was that the instance might have been in mount mode. I tried it on a database in mount stage and I got the error.

SQL> shutdown abort
ORACLE instance shut down.
SQL> startup mount;
ORACLE instance started.

Total System Global Area 404723928 bytes
Fixed Size 735448 bytes
Variable Size 234881024 bytes
Database Buffers 167772160 bytes
Redo Buffers 1335296 bytes
Database mounted.
SQL> desc dba_tablespaces
ERROR:
ORA-04043: object dba_tablespaces does not exist

I was expecting to be able to see the view after opening the database, but...

SQL> alter database open;
Database altered.

SQL> desc dba_tablespaces
ERROR:
ORA-04043: object dba_tablespaces does not exist

I could not query it either.

SQL> select * from dba_tablespaces;
select * from dba_tablespaces
*
ERROR at line 1:
ORA-00942: table or view does not exist

A quick search in Metalink returned note 296235.1 which refers to the bug 2365821 and says that if you describe any dba_* view in mount mode you cannot describe the same view even after opening the database. The only solution is to restart the database.

SQL> shutdown abort
ORACLE instance shut down.
SQL> startup mount
ORACLE instance started.

Total System Global Area 404723928 bytes
Fixed Size 735448 bytes
Variable Size 234881024 bytes
Database Buffers 167772160 bytes
Redo Buffers 1335296 bytes
Database mounted.
SQL> alter database open;

Database altered.

SQL> desc dba_tablespaces
Name Null? Type
----------------------------------------- -------- ----------------------------
TABLESPACE_NAME NOT NULL VARCHAR2(30)
BLOCK_SIZE NOT NULL NUMBER
INITIAL_EXTENT NUMBER
NEXT_EXTENT NUMBER
MIN_EXTENTS NOT NULL NUMBER
MAX_EXTENTS NUMBER
PCT_INCREASE NUMBER
MIN_EXTLEN NUMBER
STATUS VARCHAR2(9)
CONTENTS VARCHAR2(9)
LOGGING VARCHAR2(9)
FORCE_LOGGING VARCHAR2(3)
EXTENT_MANAGEMENT VARCHAR2(10)
ALLOCATION_TYPE VARCHAR2(9)
PLUGGED_IN VARCHAR2(3)
SEGMENT_SPACE_MANAGEMENT VARCHAR2(6)
DEF_TAB_COMPRESSION VARCHAR2(8)

I was not aware of this behaviour till now. I tested this on 9.2 but the bug seems not fixed in any version.

Wednesday, August 29, 2007

Unique indexes on materialized views

One of the uses of materialized views is replication. Mviews can be used to replicate a table to another database to prevent users from accessing several databases through database links. This can improve the performance of queries which frequently access that table by removing the latency of the database link.

Today the refresh job of one of the mviews we use for this purpose started getting "ORA-00001 unique constraint violated" error. This was an mview with a unique index on it. After a simple search in Metalink I have found the reason of this behavior.

I will provide a simple test case to reproduce the problem.


SQL> select * from v$version;

BANNER
----------------------------------------------------------------
Oracle9i Enterprise Edition Release 9.2.0.8.0 - 64bit Production
PL/SQL Release 9.2.0.8.0 - Production
CORE 9.2.0.8.0 Production
TNS for Solaris: Version 9.2.0.8.0 - Production
NLSRTL Version 9.2.0.8.0 - Production

SQL> create table master as select rownum r,'ROW'||rownum line from all_objects where rownum<=5;
Table created.

SQL> alter table master add primary key(r);


Table altered.

SQL> create unique index master_ind on master(line);

Index created.

SQL> create materialized view log on master with primary key;

Materialized view log created.

SQL> create materialized view snap refresh fast with primary key as select * from master;

Materialized view created.

SQL> create unique index snap_ind on snap(line);

Index created.

SQL> select * from master;

R LINE
---------- -------------------------------------------
1 ROW1
2 ROW2
3 ROW3
4 ROW4
5 ROW5

After these steps we have now one master table, an mview log on it and a fast refreshable mview of the master table. Now we make some updates to the master table to switch the LINE column's data of two rows.

SQL> update master set line='DUMMY' where r=1;

1 row updated.

SQL> update master set line='ROW1' where r=2;

1 row updated.

SQL> update master set line='ROW2' where r=1;

1 row updated.

SQL> commit;

Commit complete.

SQL> select * from master;

R LINE
---------- -------------------------------------------
1 ROW2
2 ROW1
3 ROW3
4 ROW4
5 ROW5

Now when we try to refresh the mview we get the error.

SQL> exec dbms_mview.refresh('SNAP');
BEGIN dbms_mview.refresh('SNAP'); END;

*
ERROR at line 1:
ORA-12008: error in materialized view refresh path
ORA-00001: unique constraint (YAS.SNAP_IND) violated
ORA-06512: at "SYS.DBMS_SNAPSHOT", line 820
ORA-06512: at "SYS.DBMS_SNAPSHOT", line 877
ORA-06512: at "SYS.DBMS_SNAPSHOT", line 858
ORA-06512: at line 1

If we drop the unique index and recreate it as nonunique we can refresh the mview without errors.

SQL> drop index snap_ind;

Index dropped.

SQL> create index snap_ind on snap(line);

Index created.

SQL> exec dbms_mview.refresh('SNAP');

PL/SQL procedure successfully completed.

SQL> select * from snap;

R LINE
---------- -------------------------------------------
1 ROW2
2 ROW1
3 ROW3
4 ROW4
5 ROW5

The reason of this error is explained in Metalink note 284101.1. According to that note the refresh mechanism may temporarily leave the data in an inconsistent state during the refresh. The modifications are not made with the same order as the modifications to the master table. Because of this creating unique indexes, foreign keys or primary keys on mviews can cause problems in the refresh process. This totally makes sense as it is better to keep the integrity checks on the master table rather than the mview. An mview is just a copy of the master table, so if we define the integrity constraints on the master table the mview will take care of itself.

This behaviour reproduces in 10G also.

Wednesday, August 15, 2007

Invisible indexes in 11G

11G has a new feature called Invisible Indexes. An invisible index is invisible to the optimizer as default. Using this feature we can test a new index without effecting the execution plans of the existing sql statements or we can test the effect of dropping an index without dropping it.

We can create an index as invisible or we can alter an index to become invisible.


YAS@11G>drop table t;

Table dropped.

YAS@11G>create table t as select * from all_objects;

Table created.

YAS@11G>create index t_ind on t(object_name) invisible;

Index created.

USER_INDEXES has a new column named VISIBILITY to indicate whether the index is visible or not.

YAS@11G>select index_name,VISIBILITY from user_indexes where index_name='T_IND';

INDEX_NAME VISIBILIT
------------------------------ ---------
T_IND INVISIBLE

There is a new initialization parameter modifiable at system or session level called OPTIMIZER_USE_INVISIBLE_INDEXES. This parameter is FALSE as default, meaning the optimizer will not consider invisible indexes when evaluating alternative execution plans.

YAS@11G>show parameter visible

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
optimizer_use_invisible_indexes boolean FALSE

We can alter an existing index to become invisible or visible.

YAS@11G>alter index t_ind visible;

Index altered.

YAS@11G>select index_name,VISIBILITY from user_indexes where index_name='T_IND';

INDEX_NAME VISIBILIT
------------------------------ ---------
T_IND VISIBLE

YAS@11G>alter index t_ind invisible;

Index altered.

YAS@11G>select index_name,VISIBILITY from user_indexes where index_name='T_IND';

INDEX_NAME VISIBILIT
------------------------------ ---------
T_IND INVISIBLE

We can see that the invisible index is not considered by the optimizer using a 10053 trace.

YAS@11G>alter session set events '10053 trace name context forever, level 1';

Session altered.

YAS@11G>explain plan for select * from t where object_name='STANDARD';

Explained.

In the trace file the index is marked as UNUSABLE and is not considered.

BASE STATISTICAL INFORMATION
***********************
Table Stats::
Table: T Alias: T (NOT ANALYZED)
#Rows: 80129 #Blks: 981 AvgRowLen: 100.00
Index Stats::
Index: T_IND Col#: 2
LVLS: 2 #LB: 323 #DK: 38565 LB/K: 1.00 DB/K: 1.00 CLUF: 33236.00
UNUSABLE

And the plan is:

YAS@11G>explain plan for select * from t where object_name='STANDARD';

Explained.

YAS@11G>select * from table(dbms_xplan.display);

PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------------------
Plan hash value: 1601196873

--------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 11 | 1738 | 268 (1)| 00:00:04 |
|* 1 | TABLE ACCESS FULL| T | 11 | 1738 | 268 (1)| 00:00:04 |
--------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

1 - filter("OBJECT_NAME"='STANDARD')

Note
-----
- dynamic sampling used for this statement

17 rows selected.

We can change the parameter OPTIMIZER_USE_INVISIBLE_INDEXES to see if the index will be used.

YAS@11G>alter session set optimizer_use_invisible_indexes=true;

Session altered.

YAS@11G>explain plan for select * from t where object_name='STANDARD';

Explained.

YAS@11G>select * from table(dbms_xplan.display);

PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------------------
Plan hash value: 1376202287

-------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 2 | 202 | 4 (0)| 00:00:01 |
| 1 | TABLE ACCESS BY INDEX ROWID| T | 2 | 202 | 4 (0)| 00:00:01 |
|* 2 | INDEX RANGE SCAN | T_IND | 2 | | 3 (0)| 00:00:01 |
-------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

2 - access("OBJECT_NAME"='STANDARD')

14 rows selected.

YAS@11G>alter session set optimizer_use_invisible_indexes=false;

Session altered.

If we make the index visible we see that the index is considered and used.

YAS@11G>alter index t_ind visible;

Index altered.

YAS@11G>explain plan for select * from t where object_name='STANDARD';

Explained.

YAS@11G>select * from table(dbms_xplan.display);

PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------------------
Plan hash value: 1376202287

-------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 2 | 316 | 4 (0)| 00:00:01 |
| 1 | TABLE ACCESS BY INDEX ROWID| T | 2 | 316 | 4 (0)| 00:00:01 |
|* 2 | INDEX RANGE SCAN | T_IND | 2 | | 3 (0)| 00:00:01 |
-------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

2 - access("OBJECT_NAME"='STANDARD')

Note
-----
- dynamic sampling used for this statement

18 rows selected.

Trying to gather statistics on an invisible index using either gather_index_stats or gather_table_stats with the cascade option gets an error.

YAS@11G>alter index t_ind invisible;

Index altered.

YAS@11G>exec dbms_stats.gather_index_stats(user,'T_IND');
BEGIN dbms_stats.gather_index_stats(user,'T_IND'); END;

*
ERROR at line 1:
ORA-00904: : invalid identifier
ORA-06512: at "SYS.DBMS_STATS", line 14037
ORA-06512: at "SYS.DBMS_STATS", line 14060
ORA-06512: at line 1

YAS@11G>exec dbms_stats.gather_table_stats(user,'T',cascade=>true);
BEGIN dbms_stats.gather_table_stats(user,'T',cascade=>true); END;

*
ERROR at line 1:
ORA-00904: : invalid identifier
ORA-06512: at "SYS.DBMS_STATS", line 17806
ORA-06512: at "SYS.DBMS_STATS", line 17827
ORA-06512: at line 1

YAS@11G>alter index t_ind visible;

Index altered.

YAS@11G>exec dbms_stats.gather_index_stats(user,'T_IND');

PL/SQL procedure successfully completed.

YAS@11G>exec dbms_stats.gather_table_stats(user,'T',cascade=>true);

PL/SQL procedure successfully completed.

I did not search if this is a bug or intended behaviour but we need a way to collect statistics for an invisible index, otherwise how can we test if it will be used or not? There is a way to collect statistics for an invisible index, which is to set the parameter OPTIMIZER_USE_INVISIBLE_INDEXES.

YAS@11G>alter session set optimizer_use_invisible_indexes=true;

Session altered.

YAS@11G>exec dbms_stats.gather_index_stats(user,'T_IND');

PL/SQL procedure successfully completed.

YAS@11G>alter session set optimizer_use_invisible_indexes=false;

Session altered.

Tuesday, August 14, 2007

Bind variable peeking in 11G

In one of my recent posts I mentioned about Gregory Guillou's posts about bind variable peeking in 11G. I did a simple test and research on this.

First here is the test case.


YAS@11G>create table t(id number,name varchar2(30));

Table created.

YAS@11G>insert into t select mod(rownum,2),object_name from all_objects;

64901 rows created.

YAS@11G>commit;

Commit complete.

YAS@11G>create index t_ind on t(id);

Index created.

YAS@11G>drop table t;

Table dropped.

YAS@11G>create table t(id number,name varchar2(30));

Table created.

YAS@11G>insert into t select mod(rownum,2),object_name from all_objects;

64901 rows created.

YAS@11G>insert into t select 2,'test' from dual;

1 row created.

YAS@11G>create index t_ind on t(id);

Index created.

YAS@11G>exec dbms_stats.gather_table_stats(user,'T',cascade=>true,method_opt=>'for columns id size 254');

PL/SQL procedure successfully completed.

Now we have a table with three distinct values in its ID column, values 0 and 1 have more than 30,000 rows each and value 2 has only one row.

YAS@11G>var v_id number;
YAS@11G>exec :v_id := 0;

PL/SQL procedure successfully completed.

YAS@11G>set lines 200
YAS@11G>column PLAN_TABLE_OUTPUT format a100
YAS@11G>set pages 50
YAS@11G>select count(name) from t where id = :v_id;

COUNT(NAME)
-----------
32450

YAS@11G>select * from table(dbms_xplan.display_cursor);

PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------------------
SQL_ID gsmm31bu4zyca, child number 0
-------------------------------------
select count(name) from t where id = :v_id

Plan hash value: 2966233522

---------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
---------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | | 103 (100)| |
| 1 | SORT AGGREGATE | | 1 | 20 | | |
|* 2 | TABLE ACCESS FULL| T | 32433 | 633K| 103 (1)| 00:00:02 |
---------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

2 - filter("ID"=:V_ID)

19 rows selected.




A plan using a full table scan is used for this value. Prior to 11G this plan would be fixed and used for all other bind variables regardless of their value if you have CURSOR_SHARING=EXACT, if cursor_sharing is not set to EXACT things are different.
In 11G if we run the same sql with a different value things change.

YAS@11G>exec :v_id := 2;

PL/SQL procedure successfully completed.

YAS@11G>select count(name) from t where id = :v_id;

COUNT(NAME)
-----------
1

YAS@11G>select * from table(dbms_xplan.display_cursor);

PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------------------
SQL_ID gsmm31bu4zyca, child number 1
-------------------------------------
select count(name) from t where id = :v_id

Plan hash value: 2602990223

--------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | | 2 (100)| |
| 1 | SORT AGGREGATE | | 1 | 20 | | |
| 2 | TABLE ACCESS BY INDEX ROWID| T | 1 | 20 | 2 (0)| 00:00:01 |
|* 3 | INDEX RANGE SCAN | T_IND | 1 | | 1 (0)| 00:00:01 |
--------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

3 - access("ID"=:V_ID)

20 rows selected.


Notice that I had to run this sql several times for this plan to change. After several runs of the same statement with the value 2 the plan changes to an index range scan. Why is this so?

This is a change in behaviour in bind variable peeking in 11G described here. No need to rephrase what the documentation says here, just to quote:

"To ensure the optimal choice of cursor for a given bind value, Oracle Database uses bind-aware cursor matching. The system monitors the data access performed by the query over time, depending on the bind values. If bind peeking takes place, and a histogram is used to compute selectivity of the predicate containing the bind variable, then the cursor is marked as a bind-sensitive cursor. Whenever a cursor is determined to produce significantly different data access patterns depending on the bind values, that cursor is marked as bind-aware, and Oracle Database will switch to bind-aware cursor matching to select the cursor for that statement. When bind-aware cursor matching is enabled, plans are selected based on the bind value and the optimizer's estimate of its selectivity. With bind-aware cursor matching, it is possible that a SQL statement with user-defined bind variable will have multiple execution plans, depending on the bind values."


The view V$SQL_CS_STATISTICS gives information about the cursor statistics. We can use that to see the effect of this change.

YAS@11G>alter system flush shared_pool;

System altered.

YAS@11G>exec :v_id := 0;

PL/SQL procedure successfully completed.

YAS@11G>select /* test */ count(name) from t where id = :v_id;

COUNT(NAME)
-----------
32450

YAS@11G>select sql_text,sql_id from v$sql where upper(sql_text) like 'SELECT%TEST%';

SQL_TEXT
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID
-------------
select /* test */ count(name) from t where id = :v_id
12dcwm6639zmr

select sql_text,sql_id from v$sql where upper(sql_text) like 'SELECT%TEST%'
51hmrnzmuaswy


YAS@11G>select child_number,BIND_SET_HASH_VALUE,PEEKED,EXECUTIONS,ROWS_PROCESSED,BUFFER_GETS
2 from V$SQL_CS_STATISTICS
3 where sql_id='12dcwm6639zmr';

CHILD_NUMBER BIND_SET_HASH_VALUE P EXECUTIONS ROWS_PROCESSED BUFFER_GETS
------------ ------------------- - ---------- -------------- -----------
0 1475158189 Y 1 32451 380

YAS@11G>exec :v_id := 2;

PL/SQL procedure successfully completed.

YAS@11G>select /* test */ count(name) from t where id = :v_id;

COUNT(NAME)
-----------
1

YAS@11G>select child_number,BIND_SET_HASH_VALUE,PEEKED,EXECUTIONS,ROWS_PROCESSED,BUFFER_GETS
2 from V$SQL_CS_STATISTICS
3 where sql_id='12dcwm6639zmr';

CHILD_NUMBER BIND_SET_HASH_VALUE P EXECUTIONS ROWS_PROCESSED BUFFER_GETS
------------ ------------------- - ---------- -------------- -----------
1 2064090006 Y 1 3 3
0 1475158189 Y 1 32451 380


As you can see there are two child cursors for this sql because of the different bind values.

First 11G installation

I have finished my first 11G installation using Vmware Server and Oracle Enterprise Linux 5 and created my first 11G database. The installation is no different than the 10G installation. There are a few differences in the database creation. I have created the database using dbca.

When it comes to setting the SGA and PGA sizes it asks if I want to use automatic memory management or not. Using automatic memory management we can set a single value for both SGA and PGA and the database adjusts the memory automatically. If this option is not selected it reverts back to the 10G behaviour.

There is a new option related to the security settings. 11G asks if you want to use enhanced 11G security settings. This option includes settings related to password management and auditing. If you select this option it enables these password policies in the default profile and starts auditing these privileges.

There is a new step asking if you want to enable automatic maintenance tasks. By selecting this option you enable automatic tasks such as statistics gathering, automatic segment advisor and sql tuning advisor.

Other than these the database creation is like 10G.

Friday, August 10, 2007

11G excitement in the community

Today when I was walking through the OraNA feed I saw that nearly all of the posts were about 11G being available for download. 11G is starting to get hot it seems. I am sure we will start seeing lots of posts about the new features soon.

Gregory Guillou of the Pythian Group has two posts about bind variable peeking in 11G, here and here. This is an interesting change in the cursor sharing behavior. I will test this and see how it behaves myself if I can finish my download. It is going really slow for now.

By the way the zip file is more than twice the size of the file for 10G R2. 10G R2 installation zip file was about 700MB while this is 1.7GB.

11G Available for Linux

11G for Linux is available for download here and I have just started downloading it.

At about the same time the first three parts of The Top New Features series by Arup Nanda has been published: database replay, partitioning and schema management. You can subscribe to this series' rss feed here.

Friday, July 13, 2007

Modifying sprepins.sql for Custom Statspack Reports III

Rows per Exec

There is a section titled "SQL ordered by Gets" in statspack reports. This section reports the sql statements performing most of the buffer gets.

CPU Elapsd
Buffer Gets Executions Gets per Exec %Total Time (s) Time (s) Hash Value
--------------- ------------ -------------- ------ -------- --------- ----------


One important thing missing from these columns is the number of rows processed per execution for each sql statement. With a simple change we can add a column named "rows per exec" showing this information. By knowing how many rows the sql processes I can decide if the number of buffer gets is acceptable or not. If I have a sql statement processing a few rows but performns lots of buffer gets then I need to see if something can be done about that sql.


If the sql involves calculating aggregates like group by statements then this information may not be that useful, you need to do a more detailed analysis of the sql statement. But for simple sql statements with no aggregates this can be helpful.


After this change that section looks like the following:


CPU Elapsd
Buffer Gets Executions Gets per Exec Rows per Exec %Total Time (s) Time (s) Hash Value
--------------- ------------ -------------- -------------- ------ -------- --------- ----------


To make this change find this line in sqprepins.sql.


-- SQL statements ordered by buffer gets


Remove the lines between that line and this line.


-- SQL statements ordered by physical reads


In between these two lines insert the following code. Notice the lines in bold.

ttitle lef 'SQL ordered by Gets for ' -
'DB: ' db_name ' Instance: ' inst_name ' '-
'Snaps: ' format 999999 begin_snap ' -' format 999999 end_snap -
skip 1 -
'-> End Buffer Gets Threshold: ' ebgt -
skip 1 -
'-> Note that resources reported for PL/SQL includes the ' -
'resources used by' skip 1 -
' all SQL statements called within the PL/SQL code. As ' -
'individual SQL' skip 1 -
' statements are also reported, it is possible and valid ' -
'for the summed' skip 1 -
' total % to exceed 100' -
skip 2;

-- Bug 1313544 requires this rather bizarre SQL statement
set lines 95
set underline off;
col aa format a196 heading -
' CPU Elapsd| Buffer Gets Executions Gets per Exec Rows per Exec %Total Time (s) Time (s) Hash Value |--------------- ------------ -------------- -------------- ------ -------- --------- ----------'

column hv noprint;
break on hv skip 1;

select aa, hv
from ( select /*+ ordered use_nl (b st) */
decode( st.piece
, 0
, lpad(to_char((e.buffer_gets - nvl(b.buffer_gets,0))
,'99,999,999,999')
,15)||' '||
lpad(to_char((e.executions - nvl(b.executions,0))
,'999,999,999')
,12)||' '||
lpad((to_char(decode(e.executions - nvl(b.executions,0)
,0, to_number(null)
,(e.buffer_gets - nvl(b.buffer_gets,0)) /
(e.executions - nvl(b.executions,0)))
,'999,999,990.0'))
,14) ||' '||
--Reports rows processed per exec
lpad((to_char(100*(e.rows_processed - nvl(b.rows_processed,0))/(e.executions - nvl(b.executions,0))
,'999,999,990.0'))
, 14)
||' '||
lpad((to_char(100*(e.buffer_gets - nvl(b.buffer_gets,0))/:gets
,'990.0'))
, 6) ||' '||
lpad( nvl(to_char( (e.cpu_time - nvl(b.cpu_time,0))/1000000
, '9990.00')
, ' '),8) || ' ' ||
lpad( nvl(to_char( (e.elapsed_time - nvl(b.elapsed_time,0))/1000000
, '99990.00')
, ' '),9) || ' ' ||
lpad(e.hash_value,10)||''||
decode(e.module,null,st.sql_text
,rpad('Module: '||e.module,95)||st.sql_text)
, st.sql_text) aa
, e.hash_value hv
from stats$sql_summary e
, stats$sql_summary b
, stats$sqltext st
where b.snap_id(+) = :bid
and b.dbid(+) = e.dbid
and b.instance_number(+) = e.instance_number
and b.hash_value(+) = e.hash_value
and b.address(+) = e.address
and b.text_subset(+) = e.text_subset
and e.snap_id = :eid
and e.dbid = :dbid
and e.instance_number = :inst_num
and e.hash_value = st.hash_value
and e.text_subset = st.text_subset
and st.piece < &&num_rows_per_hash
and e.executions > nvl(b.executions,0)
order by (e.buffer_gets - nvl(b.buffer_gets,0)) desc, e.hash_value, st.piece
)
where rownum < &&top_n_sql;



Rollback Segments

The sections named "Rollback Segment Stats" and "Rollback Segment Storage" provides information about individual rollback segments. The first one gives information like undo bytes written, wraps, extends, shrinks, etc... The second one gives information about the average, maximum and optimum sizes of the rollback segments. I do not remember an occasion where I used these sections for databases with automatic undo management. So, I remove these sections as I do not them for automatic undo. This change reduces my report file size from about 500KB to about 100KB. This is not a major change but helps me keep unused data out of the report.

To make this change find the lines "-- Rollback segment" and "-- Undo Segment" and comment or remove the lines between them.


The last three posts were about modifications to sprepins.sql for custom statspack reports. Actually there are more changes I use to customize my reports to my needs but I think these three posts can help you in starting if you want to customize sprepins.sql.


Tuesday, May 29, 2007

Modifying sprepins.sql for Custom Statspack Reports II

My last post was about modifying $ORACLE_HOME/rdbms/admin/sprepins.sql which is the script that produces the statspack report.

Another change I make in my sprepins.sql is about the section "Top 5 Timed Events". In 9.2 this section lists the wait events and CPU time and their percentages contributing to the database time. This way we can see which wait event caused most of the waits for that period or if it is CPU time that takes most of the time. 8.1.7 has a section titled "Top 5 Wait Events" and does not include CPU time in this section. I originally made this change for 8.1.7 to include CPU time in that section but I am still using it in 9.2 with a few changes. This last version is tested in 9.2.

In this modification I divide the CPU time into 3 parts as described in Metalink note 232443.1. as "parse time cpu", "recursive cpu usage" and "cpu other". "parse time cpu" is the time spent in parsing, "recursive cpu usage" is the time spent in recursive calls such as plsql calls from sql and "cpu other" is the remaining cpu time. This way when I look at the Top 10 Timed Events section (I list 10 events not 5) I can see which part consumes more time and if it is a problem or not.

After this change you will have a new section titled "Top 10 Timed Events" like below.


Top 10 Timed Events
~~~~~~~~~~~~~~~~~~~ Wait
Event Time (s) % TOTAL TIME
-------------------------------------------- ------------ ------------
SQL*Net message from dblink 4,700 36.01
cpu other 2,928 22.43
recursive cpu usage 2,479 19
SQL*Net more data to client 587 4.49
enqueue 456 3.49
sbtwrite2 372 2.85
async disk IO 354 2.71
log file parallel write 261 2
db file sequential read 244 1.87
single-task message 188 1.44


To make this change find the following lines in sprepins.sql.

set heading on;
repfooter left -
'-------------------------------------------------------------';

--
-- Top Wait Events


Just above this add the following lines.

----------------------------------------TOP 10 TIMED EVENTS--------------------------------------
set heading on;
repfooter left -

col event format a44 heading 'Top 10 Timed Events|~~~~~~~~~~~~~~~~~~~|Event';
col time_waited format 999,999,990 heading 'Wait|Time (s)' just c;
col pct format 999.99 heading '% Total|Time';

SELECT NAME "EVENT",VALUE "TIME_WAITED",ROUND(PCT_TOTAL*100,2) "% TOTAL TIME"
FROM (SELECT NAME, VALUE, ratio_to_report (VALUE) OVER () pct_total
FROM (SELECT s1.NAME, (s1.VALUE - s2.VALUE)/100 VALUE
FROM stats$sysstat s1, stats$sysstat s2
WHERE s1.NAME IN
('recursive cpu usage',
'parse time cpu')
AND s1.NAME = s2.NAME
AND s1.snap_id = :eid
AND s2.snap_id = :bid
UNION ALL
SELECT 'cpu other', diff/100
FROM (SELECT NAME, VALUE,
VALUE
- (LEAD (VALUE, 1) OVER (ORDER BY NAME))
- (LEAD (VALUE, 2) OVER (ORDER BY NAME))
diff
FROM (SELECT s1.NAME, s1.VALUE - s2.VALUE VALUE
FROM stats$sysstat s1, stats$sysstat s2
WHERE s1.NAME IN
('recursive cpu usage',
'parse time cpu',
'CPU used by this session'
)
AND s1.NAME = s2.NAME
AND s1.snap_id = :eid
AND s2.snap_id = :bid))
WHERE NAME = 'CPU used by this session'
UNION ALL
SELECT event, time_waited
FROM (SELECT s1.event,
(s1.time_waited_micro - nvl(s2.time_waited_micro,0))/1000000 time_waited,
ratio_to_report ( s1.time_waited_micro
- nvl(s2.time_waited_micro,0)
) OVER () pct
FROM stats$system_event s1, stats$system_event s2
WHERE s1.event = s2.event
AND s1.snap_id = :eid
AND s2.snap_id = :bid
AND s1.event NOT IN (SELECT event
FROM stats$idle_event
)))
ORDER BY pct_total DESC)
WHERE ROWNUM <= 10; ---------------------------------------END TOP 10 TIMED EVENTS-------------------------


With the help of this section I can see what the top 10 events are including the parts of CPU time and drill down from there to find the cause.

The next post will be about adding a "Rows Processed" column to the "SQL ordered by Gets" section and the removal of the sections "Rollback Segment Stats" and "Rollback Segment Storage" for databases with automatic undo management.

Thursday, May 24, 2007

Modifying sprepins.sql for Custom Statspack Reports I

I use STATSPACK extensively for comparing two time periods, comparing a time period to a baseline, monitoring trends and diagnosing performance problems. It provides important data about the database performance. The code producing statspack reports is open and you can modify it to your needs.

You run $ORACLE_HOME/rdbms/admin/spreport.sql to produce statspack reports. spreport.sql calls sprepins.sql which is the script that produces the report. You can easily modify sprepins.sql to add, remove and change sections in the report. By sections I mean parts in the report that start with headers like "SQL ordered by Gets", "Instance Activity Stats", "Load Profile", etc... I use a modified sprepins.sql to produce statspack reports that suit my needs. I want to share those modifications in a few posts here. You can use these modifications, modify them to your needs or even introduce new changes like these. Note that these changes are tested on 9.2 only.

The first two modifications I made are; entering number of days to list snapshots for and using a standard name for report outputs.

Number of Days

When spreport.sql is run it prints all the available statspack snapshots and asks for a begin snapshot id and an end snapshot id to produce a report that outputs the delta values between these snapshots. If you keeps days, weeks or even months of data in the PERFSTAT schema the listing of all snapshots can take time and you need to wait for a lot of pages to flow. To prevent this I add a new prompt that asks the number of days to list snapshots for (awrrpt.sql which produces the AWR report in 10G does this also). To make this change find the line with this sentence in sprepins.sql.

-- Ask for the snapshots Id's which are to be compared

This is the part where it lists the snapshots. Above this line put these lines.


set termout on
prompt
prompt Specify Number of Days
prompt ~~~~~~~~~~~~~~~~~~~~~~
accept num_days number default 1 prompt 'Enter number of days:'


This will ask you to enter the number of days to list and put your input into the variable num_days. If you do not enter something the default is 1 day. Now change the select statement after these lines to select only the snapshots for num_days number of days. The bold line below is the line you need to add.

select to_char(s.startup_time,' dd Mon "at" HH24:mi:ss') instart_fmt
, di.instance_name inst_name
, di.db_name db_name
, s.snap_id snap_id
, to_char(s.snap_time,'dd Mon YYYY HH24:mi') snapdat
, s.snap_level lvl
, substr(s.ucomment, 1,60) commnt
from stats$snapshot s
, stats$database_instance di
where s.dbid = :dbid
and di.dbid = :dbid
and s.instance_number = :inst_num
and di.instance_number = :inst_num
and di.dbid = s.dbid
and di.instance_number = s.instance_number
and di.startup_time = s.startup_time
and s.snap_time>=trunc(sysdate-&&num_days+1)
order by db_name, instance_name, snap_id;


Now if you save sprepins.sql and run spreport.sql it will ask the number of days:

Specify Number of Days
~~~~~~~~~~~~~~~~~~~~~~
Enter number of days:


After that it will print the snapshots for that number of days and you can continue as usual by entering snapshot ids.

Report Names

After entering the begin snapshot id and the end snapshot id sprepins.sql asks you for a report name and it will use a default report name like sp_1_2.lst if you do not enter a name (where 1 is the begin snapshot id and 2 is the end snapshot id).

To produce a standard report name I remove the report name prompt and use a default name. With this modification I have report names like PROD_22_05_2007_13_30__13_45.LST where PROD is the instance name. This way I can quickly understand which instance and which time period the report is produced for. This name tells that this report is for the instance PROD, reporting the period between 22-MAY-2007 13:30 and 22-MAY-2007 13:45.

To make this change remove or comment the following lines in sprepins.sql.

prompt
prompt Specify the Report Name
prompt ~~~~~~~~~~~~~~~~~~~~~~~
prompt The default report file name is &dflt_name.. To use this name,
prompt press to continue, otherwise enter an alternative.

set heading off;
column report_name new_value report_name noprint;
select 'Using the report name ' || nvl('&&report_name','&dflt_name')
, nvl('&&report_name','&dflt_name') report_name
from sys.dual;
spool &report_name;

In place of these lines enter the following.

column begin_time new_value begin_time noprint;
column end_time new_value end_time noprint;

select to_char(snap_time,'DD_MM_YYYY_HH24_MI') begin_time
from stats$snapshot
where snap_id=&begin_snap;

select to_char(snap_time,'_HH24_MI') end_time
from stats$snapshot
where snap_id=&end_snap;

set escape on
spool &&inst_name\_&begin_time\_&end_time;


Be aware that if you are taking a report for a period spanning two days the report name will be misleading as I only take the hour-minute part for the end snapshot. This name standardization makes keeping and referencing old reports easier.

The next post will be about changing the "Top 10 Timed Events" section of the statspack report.

Monday, May 21, 2007

Performance of Autotrace

I have been having some performance problems when using autotrace in some production systems. I finally had time to check out why.

I enabled sql trace in my autotrace session to see what was causing the slow response. As you may already know setting autotrace on opens another session other than the session you are executing your query in. This second session reports statistics of the main session which executes your query. This second session is the one I have enabled sql trace for.

In the original session, I set the timing on to see the elapsed times and run a simple query which returns immediately.

SQL> set timing on
SQL> select sid from v$mystat where rownum=1;

SID
----------
20892

Elapsed: 00:00:00.00
SQL> select * from dual;

D
-
X

Elapsed: 00:00:00.00

Now I set autotrace on.

SQL> set autot trace

This opens another session for me. You can now query v$session and find out that you have 2 sessions, one with sid=20892 which is the main session and another one with a different sid. I enable sql trace for that other session to see what it is doing. From another session run the following to enable sql trace.

SQL> exec dbms_system.SET_SQL_TRACE_IN_SESSION(133,9526,true);

PL/SQL procedure successfully completed.


Now from the main session I execute the same simple query again and set autotrace off after that.


SQL> select * from dual;

Elapsed: 00:01:14.08

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE
1 0 TABLE ACCESS (FULL) OF 'DUAL'




Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
3 consistent gets
0 physical reads
0 redo size
205 bytes sent via SQL*Net to client
273 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
1 rows processed

SQL> set autot off
Look at the elapsed time! It is more than 1 minutes with autotrace on. Now when I look at the trace file produced I see that the sql running this long is the following.


SELECT PT.VALUE
FROM
SYS.V_$SESSTAT PT WHERE PT.SID=:1 AND PT.STATISTIC# IN (7,40,41,42,115,239,
240,241,245,246) ORDER BY PT.STATISTIC#


call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 2 0.00 0.00 0 0 0 0
Execute 2 0.00 0.00 0 0 0 0
Fetch 4 73.41 73.03 0 0 0 20
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 8 73.41 73.03 0 0 0 20

Misses in library cache during parse: 0
Optimizer goal: CHOOSE
Parsing user id: 32

Rows Row Source Operation
------- ---------------------------------------------------
10 SORT ORDER BY
10 FILTER
10 FIXED TABLE FULL X$KSUSESTA
1 FIXED TABLE FULL X$KSUSGIF
The view V_$SESSTAT is based on two fixed tables, named as X$KSUSESTA and X$KSUSGIF. X$KSUSESTA is a large object holding the statistics for all sessions. So, if you have thousands of sessions it may take a long time to query this fixed table and autotrace suffers from that. Here is the number of rows from this view in my instance.

SQL> select count(*) from X$KSUSESTA;

COUNT(*)
----------
7000000

Monday, April 16, 2007

Parsing in remote databases

I was trying to decrease the number of parses for a 9.2.0.7 production database when I realized that the top parsing sql was coming from another database through a database link. This sql has a parse/execute ratio of 1 meaning it is parsed every time it is called. I have searched the bug database and found that there is a bug about parsing when you are joining a local table with a remote table, the bug number is 4913460. For those who are using db links extensively the following is a test case showing the parsing problem in remote databases.

In the remote database create a simple table:


SQL> create table remote_table
2 as select level remote_a from dual connect by level <=1000;

Table created.

SQL> create index remote_ind1 on remote_table(remote_a);

Index created.

In the local database create a simple table:

SQL> create table local_table
2 as select level local_a from dual connect by level <=5000;

Table created.

SQL> create index local_index on local_table(local_a);

Index created.

Than create a database link connecting these two databases (commands not printed here) and query the local table joining it to the remote table:

declare
v_a number;
begin
for i in 1..1000 loop
select local_a into v_a from local_table
where local_a in (select remote_a from remote_table@test.world where remote_a=i);
end loop;
end;
/

In the remote database check v$sqlarea for the part of this sql (you can first get the hash_value by querying v$sqlarea with: sql_text like '%REMOTE_TABLE%'):

SQL> COLUMN sql_text FORMAT a30
SQL> select hash_value,sql_text,executions,parse_calls from v$sqlarea where hash_value=1971931531;

HASH_VALUE SQL_TEXT EXECUTIONS PARSE_CALLS
---------- ------------------------------ ---------- -----------
1971931531 SELECT "REMOTE_A" FROM "REMOTE 1000 1000
_TABLE" "REMOTE_TABLE" WHERE "
REMOTE_A"=:1


The sql in the remote database is parsed for every execution. In the local database again, try with EXISTS:

declare
v_a number;
begin
for i in 1..1000 loop
select local_a into v_a from local_table
where local_a=i and exists (select null from remote_table@test.world where remote_a=local_table.local_a);
end loop;
end;
/

The remote database shows:

SQL> r
1* select hash_value,sql_text,executions,parse_calls from v$sqlarea where hash_value=1971931531

HASH_VALUE SQL_TEXT EXECUTIONS PARSE_CALLS
---------- ------------------------------ ---------- -----------
1971931531 SELECT "REMOTE_A" FROM "REMOTE 2000 2000
_TABLE" "REMOTE_TABLE" WHERE "
REMOTE_A"=:1

This bug will be fixed in 11.1 but has a patch on 9.2.0.8.

Tuesday, April 10, 2007

The Read Consistency Trap in Sql Tracing

There are some important things to consider when interpreting sql trace files. One of the most important ones is the read consistency trap as it is called in the documentation.

If you see different number of block reads with the same data at different times when you trace a sql statement, this can be one reason. A test case showing what this means follows:


SQL> create table t as select * from all_objects;

Table created.

SQL> alter session set sql_trace=true;

Session altered.

SQL> begin
2 for t in (select * from t) loop
3 null;
4 end loop;
5 end;
6 /

PL/SQL procedure successfully completed.

After doing a full table scan from the above session, update all rows from another session:

SQL> update t set object_name='test';

23840 rows updated.

Execute the last sql from the first session again:

SQL> /

PL/SQL procedure successfully completed.


When you get the tkprof output with aggregate=no you can see the statistics for the two executions of the same statement differently without aggregation.

tkprof test_ora_6273.trc test.txt aggregate=no


In test.txt you can see the two executions of the statement:

SELECT *
FROM
T


call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 23841 0.82 0.71 331 23844 0 23840
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 23843 0.82 0.71 331 23844 0 23840

SELECT *
FROM
T


call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 23841 1.19 1.00 0 48015 0 23840
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 23843 1.19 1.00 0 48015 0 23840

Why did the numbers in the "query" column approximately double when the number of rows, the number of fetches did not change? Because there is an active transaction on the table which updated all the rows. So in order to get a read consistent view of the table the query goes to the undo blocks as well as table blocks.

What this suggests is; it is important when to trace an application. If the application you are trying to tune is running when there are transactions on the tables it accesses then you need to trace the application in that context. If you trace it when there are no transactions on its tables, you can come to the conclusion that the number of block reads is appropriate. Or you can focus on an sql taking much of the execution time when the sql you need to tune is actually another one.

Saturday, April 07, 2007

An effect of sql_trace=true

Tom Kyte has a recent post about explain plan.

In that there is something I have not realized before that I want to share. When you execute a sql statement after you set sql_trace=true it is hard parsed again even if it is already in the shared pool.

A simple test is below.


SQL> select 'test' from dual;

'TES
----
test

SQL> alter session set sql_trace=true;

Session altered.

SQL> select 'test' from dual;

'TES
----
test

SQL> exit


The tkprof output shows that the same statement run after setting sql_trace=true is hard parsed again.


select 'test'
from
dual


call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 2 0.00 0.00 0 1 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 4 0.00 0.00 0 1 0 1

Misses in library cache during parse: 1




Compare this with the same sql run two times after setting sql_trace=true.


SQL> alter session set sql_trace=true;

Session altered.

SQL> select 'test2' from dual;

'TEST
-----
test2

SQL> r
1* select 'test2' from dual

'TEST
-----
test2


SQL> exit



select 'test2'
from
dual


call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 2 0.00 0.00 0 0 0 0
Execute 2 0.00 0.00 0 0 0 0
Fetch 4 0.00 0.00 0 2 0 2
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 8 0.00 0.00 0 2 0 2

Misses in library cache during parse: 1


Sql trace has the side effect of hard parsing every statement executed after it is enabled. If you have your sql statements already in the shared pool before enabling sql trace, be careful that you will be seeing the execution plans for your particular execution in that particular time, not the ones from the shared pool.

Wednesday, April 04, 2007

Client-side Load Balancing

A new case study has been published in Metalink about how to configure a high connect load environment. If you have an environment where bursts of logon activity appears sometimes and your tns listener is unable to cope with the demand then you can take a look at the note to see possible solutions. It is a simple note that basically describes how the tns listener works and what to when the listener cannot respond to connection requests.

There are mainly two solutions suggested:

1. Increasing the number of listeners
2. Using shared server

Increasing the number of listeners is the first logical solution that comes up. Because a single listener process can handle a limited number of requests at one time, multiple listener processes can handle many more requests. When you configure multiple listeners you also need to change the tns entries in your clients. The sample tns entry provided in that note is this:

GEORGE=
(DESCRIPTION=
(LOAD_BALANCE=ON)
(FAILOVER=ON)
(ADDRESS=(PROTOCOL=TCP)(HOST=george.us.oracle.com)(PORT=1521)
(ADDRESS=(PROTOCOL=TCP)(HOST=george.us.oracle.com)(PORT=1522)
(ADDRESS=(PROTOCOL=TCP)(HOST=george.us.oracle.com)(PORT=1523)
(ADDRESS=(PROTOCOL=TCP)(HOST=george.us.oracle.com)(PORT=1524)
(CONNECT_DATA=(SERVICE_NAME=V10R2.us.oracle.com)))

This ensures that connections are balanced between the specified ports, meaning different listener processes.

But there is one restriction that is not mentioned in the case study. If you have clients using Net8 (not higher versions) like Forms6i applications, the LOAD_BALANCE parameter does not exist, so you cannot load balance client connections using this entry. If you have such clients you need to use description lists in your tns entry as described in Metalink note 67137.1. The tns entry for load balancing in Net8 is like this:

TEST=
(DESCRIPTION_LIST=
(DESCRIPTION=
(ADDRESS=
(PROTOCOL=TCP)(HOST=host1)(PORT=1521)
)
(CONNECT_DATA=(SID=TEST))
)
(DESCRIPTION=
(ADDRESS=
(PROTOCOL=TCP)(HOST=host1)(PORT=1522)
)
(CONNECT_DATA=(SID=TEST))
)
)

Sure it is better to fix the application to correct the logon burst problem but in case you have a situation where you cannot fix the application for any reason it is good to be aware of these solutions.

Friday, November 10, 2006

Testing Blogmailr

From Lifehacker:

A new service to post to your blog by sending an e-mail is out. Blogmailr gives you an e-mail address to send your posts and allows to specify secure sender addresses to accept e-mails from.

I am sending this post using my secure Blogmailr address to see if it works.

See it for yourself at www.blogmailr.com.

Monday, September 04, 2006

Reverse Key Index

There is an option to create index entries as reversed, which is called reverse key indexes. Oracle stores the index entries as their bytes reversed, except rowids of course.

There are a few cases where reverse key indexes can help to improve performance. One is in RAC environments. If you have a column populated by an increasing sequence the new entries come to the same blocks when you are inserting rows. If you have many concurrent sessions inserting rows from different RAC instances then you will have a contention for the same index blocks between nodes. If you use reverse key indexes in this case then the new index entries will go to different blocks and contention will be reduced.

In single instance databases there is also a case where reverse key indexes can be helpful. If you have a column populated by an increasing sequence, you delete some old rows from the table and you do not do range scans on that column and you have contention issues on index blocks, reverse key indexes can be considered. The reverse key index will scatter the entries accross different blocks during inserting and your many concurrent sessions will not have index block contention issues.

If you are deleting some old rows, the blocks from a normal index on that column will have some used and some empty space in them, but they will not be put on the freelist because they are not completely free. That empty space will not be used because the sequence values are always increasing and they will not go to those old blocks because of that. You will be able to use that space for different values with reverse key indexes.

One of the things to be careful about reverse key indexes is that you cannot perform range scans on them. Because the entries are stored as reversed you lose the capability to range scan on that index.

I had a situation just fitting the reasons to use a reverse key index. We have many concurrent programs that insert into the same table. The table has a primary key column populated by an increasing sequence. There are no range scans on that column. The data is deleted time to time according to some rules which leave some old data undeleted in the table. When these programs are running statspack reports show high buffer busy waits for the index segment (More than 900,000 waits for a 30 minute period causing %85 of all buffer busy waits). Also as this database will be converted to a RAC database soon, this case seems very appropriate to use a reverse key index on the related column.

To change an existing index as a reverse key index you can use the alter index statement.

alter index indexname rebuild reverse;

After this change the index size was reduced down to 16MB from 250MB. This change got rid of the buffer busy waits on the index blocks. The program run time was reduced from about 40 minutes to about 25 minutes.

A few discussions from asktom about reverse key indexes are here, here and here.

Sunday, August 20, 2006

Effective Oracle by Design

I have finished reading Effective Oracle by Design. I consider this as one of the must reads after Oracle documentation for both dbas and developers  (Oracle documentation comes before any other book of course). It is not a beginner's book, fundamentals of sql, plsql and database administration are not the topics of the book. Tom Kyte explains how to design and build high performance applications on the Oracle database.

Designing before building an application is the most important and critical item in developing successful applications. Unfortunately it is also generally the most overlooked part. Datatypes, table types, tools to use, database features to use are critical for the application's performance and success. The application developed without performance in mind will end as a failure most probably. In this book Tom Kyte explains what the right approach he thinks is to design successful applications. What are the different types of tables available? What are the tools available? How is a sql statement processed in the database? How can one use plsql effectively? When should we use a specific database feature and when should not we use that feature? All chapters presented with lots of examples and you can test the examples provided yourself to gain more understanding of the subject.

If you are a regular reader of the asktom site as I am, you will find examples and subjects scattered around many questions and answers similar to those in the book. The examples in the book were familiar to me. But he collects all those subjects and examples in this book in a systematic way. Each chapter can be read on its own but I strongly suggest reading it line by line from the beginning to the end. It is well worth the effort and the time.

The first print year of the book is 2003. The examples are on 9.2 and 8.1.7. But do not think that this is an outdated book for 10g. Because of the above mentioned reasons the contents are also applicable to applications being developed on 10g. Tom gives his thoughts, experience, hints, tips about the right approach to developing applications. These are independent of the database version.

According to me the most important chapters are 7th (Effective Schema Design), 8th (Effective SQL) and 9th (Effective PL/SQL Programming).

Having every dba and developer in your team read this book will immediately make a significant positive effect on the applications you develop.

So, now I have two books waiting to be read. Expert Oracle Database Architecture and Cost-Based Oracle Fundamentals.

Thursday, August 17, 2006

DBA vs. Developer

A few days ago there was a short thread on asktom about the role of the developer and the dba. This is a topic that is usually discussed. Some think dbas are a blockage to stop the developers doing their work, some think dba is the king.

I have seen some dbas who think they do not need to know about plsql, analytics and other things related to development. I have seen many developers who are not aware of anything called analytics, tkprof, plsql features like bulk collect, etc. I think both are dangerous.

I have started my career as a developer writing code on Oracle Forms and plsql. I currently work as a production dba. This does not involve routine administration like backups, upgrades, installations, tablespace management, etc. only.  It involves sql tuning, reviewing the developers' code for performance improvements, suggesting physical schema design changes. Usually developers come to me to ask about a poorly performing sql or plsql procedure, about how to do something efficiently in plsql, about how to design the schemas. I can say I work as an internal consultant to developers.

Actually what I like in being this kind of a dba is the "working with developers" part, not the "production dba" part. I like to learn and study database features, sql and plsql tuning. Fortunately routine administration tasks are becoming easier and easier as new versions bring new features that are helpful to dbas and they take less amount of time in each new version.

Tom has a nice classification for dbas and developers. I consider myself as a dba/developer according to that. To be a successful dba you should know about development also, to be a successful developer you should know what the database features are, what it offers to you to do your job in an efficient way.

I cannot understand a dba who thinks that developers are a waste of time, who does not like to work with developers, who throws the code to the developers and order them to correct and tune it. As far as I have seen it is true that most developers do not know about cbo, analytics, tkprof, stuff they should know about, but the dba must take responsibility on this and try to make the developer aware of these and help them use all the features of the database appropriately. Databases are there to store data and data is there to be made available to users through programs made by developers. So it is a team work including dbas and developers to make the database useful to users.

Sunday, August 13, 2006

Back to blogging

I have not posted anything here for a long time. That was because i had lost the will to blog. I did not give up reading all the blogs on my list during that time because i find all of them very useful and fun to read. But when it came to writing and posting myself, i did not have the will to do it.

Since my last post, nothing significant has changed. I have entered the OCP exams and received my database OCP certificate. I will post about that soon. I have read some Oracle books of course, the best and the most important one being Effective Oracle by Design. I have recently bought Cost-Based Oracle Fundamentals and Expert Oracle Database Architecture. I will post about these also.

So, back to blogging...