John’s Oracle Experiences

My everyday experiences with Oracle products

Easy way of monitoring errors while using a reverse proxy

Posted by John Paul van Helvoort on November 20, 2009

When passing a website using a reverse proxy , you might encounter an 503 error when the site you try to pass on is for some reason not available.
You can monitor this behavior by implementing your own 503 reporting system.
First you take on a line that will catch these errors.

Alias /error /srv/www/htdocs/error
ErrorDocument 503 /error/503.php

After this you can put together your own 503.php script

<?php
ob_start();
header('HTTP/1.1 503');
header('Status: 503');
$error=503
?>
<html>
 <title>Example.com Page Reported <? echo $error ?></title>
 <body>
 <br>
 <center><img src=/error/images/logo.png></center>
 <br>
 <center> Technical Problem Encountered</center>
 </body>
</html>
<?
// Retrieve the URL requested
function curPageURL() {
  $isHTTPS = (isset($_SERVER["HTTPS"]) && $_SERVER["HTTPS"] == "on");
  $port = (isset($_SERVER["SERVER_PORT"]) && ((!$isHTTPS && $_SERVER["SERVER_PORT"] != "80") || ($isHTTPS && $_SERVER["SERVER_PORT"] != "443")));
  $port = ($port) ? ':'.$_SERVER["SERVER_PORT"] : '';
  $url = ($isHTTPS ? 'https://' : 'http://').$_SERVER["SERVER_NAME"].$port.$_SERVER["REQUEST_URI"];
  return $url;
}

// Fill basic variables
  $page=curPageUrl();
  $today = date("F j, Y, G:i");
  $ip=$_SERVER['REMOTE_ADDR'];

  $headers  = 'MIME-Version: 1.0' . "\r\n";
  $headers .= 'Content-type: text/html; charset=iso-8859-1' . "\r\n";
// E-mailadres to inform
  $to='webmaster@example.com';

  $subject='ERROR '.$error.' Reported on '.$page;

  $message = '
  <html>
  <body>
    <table>
      <tr>
       <td>Requested URL</td><td>'.$page.'</td>
      </tr>
      <tr>
       <td>Requested By</td><td>'.$ip.'</td>
      </tr>
      <tr>
       <td>Requested On</td><td>'.$today.'</td>
      </tr>
    </table>
    </table>
  </body>
  </html>
  ';
// Mail the error
  mail($to, $subject, $message, $headers);
?>

Offcourse you can use this same code to report on different errors reported aswell :).

Posted in Apache, Linux | Leave a Comment »

Protect your public APEX admin site when using an apache reverse proxy

Posted by John Paul van Helvoort on November 20, 2009

To catch all traffic which requests an APEX admin page like 4550 , you can include these RewriteRules in your Apache Reverse Proxy configuration.
When not set on the first entry point ( in this case the reverse proxy ), you might run into the problem that all traffic received on your back-end server seem to come from an internal adres( your reverse proxy server)
and can therefor not be filtered using the build-in security feature in APEX. So be implementing these rules your site can be made more secure again.


# Set Engine on
RewriteEngine On

# 403 error will be thrown if met
# exclude the password change page for apex users
RewriteCond %{REQUEST_URI}%{QUERY_STRING} !/(apex|builder)/f?p=4155:.*
RewriteCond %{REQUEST_URI}%{QUERY_STRING} /(apex|builder)/f?p=(4[0-9]{3}:.*)
# Making it available still from 127.0.0.1 and 10.0.0.x
RewriteCond %{REMOTE_ADDR}        !^(127\.0\.0\.1|10\.0\.0\.[0-9]{1,3})$

# Else , let request go threw
RewriteRule /(apex|builder)/ - [F]

If you want to catch these 403 errors and show a nice error message for it, you can set these lines aswell


  alias /error /srv/www/htdocs/error

  ErrorDocument 403 /error/forbidden.html

Now place a forbidden.html file in /srv/www/htdocs/error/ and you are all set.

Posted in Apache, Apex, Linux | Leave a Comment »

Check out Oracle Enterprise Manager Desktop Widgets

Posted by John Paul van Helvoort on November 12, 2009

Oracle now provides us with a desktop tool to monitor our environment using the an Oracle Enterprise Grid Control.
Simply install the desktop widget en search your favorite targets to be listed. From here you have a straight link to the target in your monitoring tool in case of any problem.

Here are some examples from after the installation on my desktop.

Target Search & Monitoring

Login screen:
Screen shot 2009-11-11 at 10.31.56 PM

Monitoring targets
Screen shot 2009-11-11 at 10.34.00 PM

High-Load Databases

Load on all databases known to EMGC.
Screen shot 2009-11-11 at 10.38.54 PM

Please visit the Oracle site and try-out there new Tools !
http://www.oracle.com/technology/products/oem/widgets/index.html

Posted in Grid Control | Leave a Comment »

Failed to start a managed process after using own SSLWallet

Posted by John Paul van Helvoort on November 11, 2009

After creating a new Oracle Wallet for my apache instance i ran into the error that the opmnctl service would not start when i changed the ssl.conf configuration to pickup the new wallet.

#SSLWallet file:C:\oracle\product\10.1.3\apache\ohs\conf\ssl.wlt\default
SSLWallet file:C:\oracle\WALLETS

Instead i got :

C:\oracle\product\10.1.3\apache\opmn\bin>opmnctl startall
opmnctl: starting opmn and all managed processes…
================================================================================
opmn id=IAS-SRV:6200
0 of 1 processes started.

ias-instance id=IAS-1
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
——————————————————————————–
ias-component/process-type/process-set:
HTTP_Server/HTTP_Server/HTTP_Server/

Error
–> Process (index=1,uid=1246497958,pid=2160)
failed to start a managed process after the maximum retry limit
Log:
C:\oracle\product\10.1.3\apache\opmn\logs\\HTTP_Server~1.log

So stop the , if any , processes again.

C:\oracle\product\10.1.3\apache\opmn\bin>opmnctl stopall
opmnctl: stopping opmn and all managed processes…

To solve this , you should check the auto-logon in your wallet to operate under these conditions.

autologon

After this start you opmn processes.

C:\oracle\product\10.1.3\apache\opmn\bin>opmnctl startall
opmnctl: starting opmn and all managed processes…

C:\oracle\product\10.1.3\apache\opmn\bin>

Posted in Apache, Application Server | Leave a Comment »

Exception in thread “main” java.lang.IndexOutOfBoundsException: No group 9

Posted by John Paul van Helvoort on November 10, 2009

Exception in thread “main” java.lang.IndexOutOfBoundsException: No group 9

While installing Oracle Webcenter Suite 10.1.3.2.0 i ran into the following error.

Command: /u00/oracle/product/10.1.3/web/jdk/bin/java -jar /u00/oracle/product/10.1.3/web/j2ee/home/admin_client.jar deployer:oc4j:opmn://xxx.helvoort.nl:6004/OC4J_WebCenter oc4jadmin -script /u00/oracle/product/10.1.3/web/portal/conf/oc4jportal_server_xml.params
Exception in thread “main” java.lang.IndexOutOfBoundsException: No group 9
at java.util.regex.Matcher.group(Matcher.java:463)
at java.util.regex.Matcher.appendReplacement(Matcher.java:730)
at java.util.regex.Matcher.replaceAll(Matcher.java:806)
at java.lang.String.replaceAll(String.java:2000)
at oracle.webdb.config.PortletContainerConfigAssistant.replacePwd(Unknown Source)
at oracle.webdb.config.PortletContainerConfigAssistant.configServerXML(Unknown Source)
at oracle.webdb.config.PortletContainerConfigAssistant.main(Unknown Source)

Please use a password without “$” as this is not escaped during configuration and therefor causing the installation to fail during a replace action.

Posted in Application Server | Leave a Comment »

Install has detected that Oracle Application Server is already installed in this Oracle Home

Posted by John Paul van Helvoort on November 9, 2009

When trying to install an application server 10.1.3.2.0 i ran into the following error.

already_installed

The error is thrown based on an earlier attempt on installing an application server. However, the product was removed successfully and should therefor not throw this error. Even when a different ORACLE_HOME is chosen , the error returns.
This behavior is caused by a corrupted Oracle Inventory as a quick peek in the file inventory.xml located in the ContentXML directory under your current oraInventory reveals that there is no such ORACLE_HOME present at this time.

As a workaround i created a dummy file called “oraInst.loc” in the “/tmp” directory containing these lines :

inventory_loc=/tmp/oraInventory
inst_group=dba

After creating this file , start your installer using

./runInstaller -invPtrloc /tmp/oraInst.loc

Your installer will continue now.

Posted in Application Server | Leave a Comment »

Generating FlashChart interactive reports on Apex fails with Flash Security Error

Posted by John Paul van Helvoort on November 6, 2009

After switching our Embedded Gateway to an Apache front-end using modplsql.
We are faced with an error while loading interactive reports using FlashChart.

As we are using a Reverse Proxy to route the traffic to our internal servers , this is actually the cause of our problem for interactive report to show successfully.

In order to overcome this behavior , we added the following lines to our $ORACLE_APACHE_HOME/ohs/modplsql/dads.conf file in between the “Location” definition:

PlsqlCGIEnvironmentList HTTP_HOST=xxx.yenlo.nl

PlsqlCGIEnvironmentList REQUEST_PROTOCOL=http

xxx.yenlo.nl being our public adres for outside and “http” being our protocol.

Restart your apache services by executing :

[apache] xxx:bin> ./opmnctl stopall
opmnctl: stopping opmn and all managed processes…
[apache] xxx:bin> ./opmnctl startall
opmnctl: starting opmn and all managed processes…

Now your FlashChart should show up successfull again !

Posted in Apex, Application Server | Leave a Comment »

Tuning your Oracle HTTP Server (Apache 2.0) to use multi-threading

Posted by John Paul van Helvoort on November 3, 2009

By default the Apache 2.0 version of Oracle HTTP Server is installed without using the HTTP “worker” method. As this is a performance enhancer , you would like to change your Apache configuration to implement the use of it.
For example , your Apex application will benefit from this as the multi-threated method with mod_plsql is supporting data connection pooling. The old fashion prefork method however is not and there for is not preferable.

Default your Apache process list will look like this after installing the Oracle HTTP Server (Apache 2.0) (10.1.3.3.0);

[apache] xxx:conf> ps -ef | grep httpd
oracle 3775 3738 0 Oct31 ? 00:00:00 /u00/oracle/product/10.1.3/apache/ohs/bin/httpd -d /u00/oracle/product/10.1.3/apache/ohs -DSSL
oracle 3779 3775 0 Oct31 ? 00:00:00 /u00/oracle/product/10.1.3/apache/ohs/bin/httpd -d /u00/oracle/product/10.1.3/apache/ohs -DSSL
oracle 3781 3775 0 Oct31 ? 00:03:48 /u00/oracle/product/10.1.3/apache/ohs/bin/httpd -d /u00/oracle/product/10.1.3/apache/ohs -DSSL
oracle 3794 3775 0 Oct31 ? 00:00:00 /u00/oracle/product/10.1.3/apache/ohs/bin/httpd -d /u00/oracle/product/10.1.3/apache/ohs -DSSL
oracle 3801 3775 0 Oct31 ? 00:00:00 /u00/oracle/product/10.1.3/apache/ohs/bin/httpd -d /u00/oracle/product/10.1.3/apache/ohs -DSSL
oracle 4042 3775 0 Oct31 ? 00:00:00 /u00/oracle/product/10.1.3/apache/ohs/bin/httpd -d /u00/oracle/product/10.1.3/apache/ohs -DSSL
oracle 4050 3775 0 Oct31 ? 00:00:00 /u00/oracle/product/10.1.3/apache/ohs/bin/httpd -d /u00/oracle/product/10.1.3/apache/ohs -DSSL
oracle 4327 3775 0 Oct31 ? 00:00:00 /u00/oracle/product/10.1.3/apache/ohs/bin/httpd -d /u00/oracle/product/10.1.3/apache/ohs -DSSL
oracle 16187 3775 0 Nov02 ? 00:00:00 /u00/oracle/product/10.1.3/apache/ohs/bin/httpd -d /u00/oracle/product/10.1.3/apache/ohs -DSSL
oracle 18433 3775 0 Nov01 ? 00:00:00 /u00/oracle/product/10.1.3/apache/ohs/bin/httpd -d /u00/oracle/product/10.1.3/apache/ohs -DSSL
oracle 22394 3775 0 Nov01 ? 00:00:00 /u00/oracle/product/10.1.3/apache/ohs/bin/httpd -d /u00/oracle/product/10.1.3/apache/ohs -DSSL
oracle 30316 3775 0 Nov02 ? 00:00:00 /u00/oracle/product/10.1.3/apache/ohs/bin/httpd -d /u00/oracle/product/10.1.3/apache/ohs -DSSL
oracle 30439 3775 0 Nov02 ? 00:00:00 /u00/oracle/product/10.1.3/apache/ohs/bin/httpd -d /u00/oracle/product/10.1.3/apache/ohs -DSSL

Here you can clearly see there is no worker process active, but instead the old fashion prefork processes are used.

The following can be done to implement the worker method which will add multi-threading to your httpd configuration.

open your opmn.xml ( located : $ORACLE_HOME/opmn/conf ) and find the following section :

    <ias-component id="HTTP_Server">
        <process-type id="HTTP_Server" module-id="OHS2">
          <module-data>
            <category id="start-parameters">
              <data id="start-mode" value="ssl-enabled"/>
            </category>
          </module-data>
          <process-set id="HTTP_Server" numprocs="1"/>
        </process-type>
      </ias-component>

Change it to ;

    <ias-component id="HTTP_Server">
        <process-type id="HTTP_Server" module-id="OHS2">
          <module-data>
            <category id="start-parameters">
              <data id="start-mode" value="ssl-enabled"/>
              <data id="mpm" value="worker"/>
            </category>
          </module-data>
          <process-set id="HTTP_Server" numprocs="1"/>
        </process-type>
      </ias-component>

After this just restart your opmn services

[apache] xxx:bin> ./opmnctl stopall
opmnctl: stopping opmn and all managed processes…
[apache] xxx:bin> ./opmnctl startall
opmnctl: starting opmn and all managed processes…

Now lets check if the worker method is implemented correctly;

[apache] xxx:bin> ps -ef | grep httpd
oracle 23556 23521 3 14:26 ? 00:00:00 /u00/oracle/product/10.1.3/apache/ohs/bin/httpd.worker -d /u00/oracle/product/10.1.3/apache/ohs -DSSL
oracle 23560 23556 0 14:26 ? 00:00:00 /u00/oracle/product/10.1.3/apache/ohs/bin/httpd.worker -d /u00/oracle/product/10.1.3/apache/ohs -DSSL
oracle 23562 23556 0 14:26 ? 00:00:00 /u00/oracle/product/10.1.3/apache/ohs/bin/httpd.worker -d /u00/oracle/product/10.1.3/apache/ohs -DSSL
oracle 23564 23556 0 14:26 ? 00:00:00 /u00/oracle/product/10.1.3/apache/ohs/bin/httpd.worker -d /u00/oracle/product/10.1.3/apache/ohs -DSSL
oracle 23565 23556 0 14:26 ? 00:00:00 /u00/oracle/product/10.1.3/apache/ohs/bin/httpd.worker -d /u00/oracle/product/10.1.3/apache/ohs -DSSL

When you are getting back the httpd.worker process you are running the multithreaded (MPM) method of the Apache 2.0 HTTP Server.

Posted in Application Server | Leave a Comment »

error while loading shared libraries: libgdbm.so.2 or libdb-3.3.so

Posted by John Paul van Helvoort on November 1, 2009

While installing the latest HTTP Server (Oracle HTTP Server (Apache 2.0) (10.1.3.3.0) ) from oracle to take advantage of the modplsql module that is provided.
The installation completed succesfully but still the httpd process could not start because of missing library dependencies.

When opening your $ORACLE_ACACHE_HOME/opmn/logs/HTTP_Server~1.log

you could find the following errors there :

——–
09/10/31 18:28:32 Start process
——–

/u00/oracle/product/10.1.3/apache/ohs/bin/apachectl startssl: execing httpd
/u00/oracle/product/10.1.3/apache/ohs/bin/httpd: error while loading shared libraries: libgdbm.so.2: cannot open shared object file: No such file or directory

——–
09/10/31 18:37:31 Start process
——–
/u00/oracle/product/10.1.3/apache/ohs/bin/apachectl startssl: execing httpd
/u00/oracle/product/10.1.3/apache/ohs/bin/httpd: error while loading shared libraries: libdb-3.3.so: cannot open shared object file: No such file or directory

In the same order you could fix this by creating symbolic links to the current shared library for each missing library.

( execute as root user )
cd /usr/lib

ln -s libgdbm.so.3.0.0 libgdbm.so.2

ln -s libdb-4.3.so libdb-3.3.so

These versions are found on “SUSE Linux Enterprise Server 10 SP2 (i586)” , your symbolic links can be different when the problem is faced on a different version operating system.
Please adjust accordingly.

( execute as root user )
cd /usr/lib

ln -s libgdbm.so.X.0.0 libgdbm.so.2

ln -s libdb-4.X.so libdb-3.3.so

Posted in Application Server, Linux | Leave a Comment »

Creating standby database 10.1.0.4.0 using OMF

Posted by John Paul van Helvoort on October 28, 2009

While creating a standby database on 10.1.0.4.0 , i ran in some trouble while performing the following RMAN statements :

RMAN> run{
ALLOCATE AUXILIARY CHANNEL dup1 DEVICE TYPE DISK
DUPLICATE TARGET DATABASE FOR STANDBY DORECOVER;
}

First a channel will be allocated on which the duplication starts. The datafiles are places in the right location. But when it comes to the point where the duplicate action executes :

contents of Memory Script:
{
switch clone datafile all;
}
executing Memory Script

The following error is thrown :

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of Duplicate Db command at 10/28/2009 19:08:41
RMAN-03015: error occurred in stored script Memory Script
RMAN-06136: ORACLE error from auxiliary database: ORA-19625: error identifying file /u02/oradata/COREF/datafile/o1_mf_system_5fyfx677_.dbf
ORA-27037: unable to obtain file status
Linux Error: 2: No such file or directory
Additional information: 3

RMAN>

Here i could see that the filename which is called for , is indeed the correct one.

After some researching i found that there is a difference in the database version in which the performed actions take place.
I did set the DB_FILE_NAME_CONVERT and LOG_FILE_NAME_CONVERT , which actually should NOT be done in rdbms version 10.1.4.0. ( other versions you should , if appropriate )
Also , when performing a duplicate when the primairy database is using OMF , you should set the standby_file_management=AUTO as this will restore datafile using %u which will be converted in the correct name to be picked up by the script.

After setting the following parameters in the auxiliary pfile, the RMAN duplicate finished successfully.

db_name = COREP (- must be set to a name of the primary db_name)
db_unique_name = COREF (- must be set to a name different from primary db_unique_name)
standby_file_management = AUTO (- must set to AUTO)
db_create_file_dest = ‘/u02/oradata’ (- must be set)

Posted in Database, RMAN | Leave a Comment »