Dynamic Loading on an SGI in Splus

B. Narasimhan
Department of Statistics
Stanford University
Stanford, CA 94305

Version of 1997/05/20

Table of Contents

Abstract

This little note tells you how to dynamically load Fortran and C programs in Splus on our SGI server where shared libraries are used.

The Problem

[*]

Consider the following fortran subroutine that takes three arguments, an array x, an integer n that indicates the length of the array and a storage area result where it should return the sum of the elements in the array. 

<Fortran subroutine>=
        subroutine sumvec(x,n,sum)
        real x(n), sum
        integer n
        sum = 0.0
        do 10 i=1,n
                sum = sum + x(i)
10      continue
        return
        end
Defines sumvec (links are to index).

How can one call such a function from Splus? Perform the following steps. 

<Steps>=
<Compile the fortran routine>
<Create a shared library>
<Load the shared library into Splus>
<Invoke the fortran routine>

Compiling the Fortran routine

[*]

Assuming that the fortran program is stored in the file foo.f do the following. 

<Compile the fortran routine>= (<-U)
rgmiller> f77 -c foo.f
Defines foo.o (links are to index).

Creating the shared library

[*]

Let us called the shared library libfoo.so. Create the shared library as follows. 

<Create a shared library>= (<-U)
rgmiller> Splus SHLIB -o libfoo.so foo.o
Defines libfoo.so (links are to index).

Shortcut

[*]

In recent versions of Splus, the two steps mentioned above can be combined into one. 

<Shortcut>=
rgmiller> Splus SHLIB -o libfoo.so foo.f

Loading the shared library

[*]

Load the shared library into Splus as follows. 

<Load the shared library into Splus>= (<-U)
> dyn.load.shared("./libfoo.so")

Invoking the routine.

[*]

The function can now be invoked from Splus. However a few words of warning before we illustrate the use. Splus stores its variables in its own format and they must be coerced into a format palatable to fortran. One uses the storage.mode function of Splus for this purpose. Table [->] from Venables and Ripley[cite vena:ripl:1994] below shows the correspondence between Splus, C and Fortran variables.

Splus storage mode --- C --- Fortran
"logical" --- long * --- LOGICAL
"integer" --- long * --- INTEGER
"single" --- float * --- REAL
"double" --- double * --- DOUBLE PRECISION
"character" --- char ** --- CHARACTER (*)
"complex" --- struct { --- DOUBLE COMPLEX
--- double re, im;}* ---
"list" --- void ** ---
Data type correspondence between Splus, C and Fortran [*]

So here is an example invocation. 

<Invoke the fortran routine>= (<-U)
> dyn.load.shared("./libfoo.so", symbols=symbol.F(sumvec))
> x <- 1:10
> storage.mode(x) <- "single"
> n <- as.integer(length(x))
> y <- 0
> storage.mode(y) <- "single"
> .Fortran("sumvec", x, n, result = y)
Defines n, x (links are to index).

The result is a list with all the arguments you passed. You can refer to the result as usual via $result.

Note carefully how the storage modes are coerced to match the single precision reals that fortran expects. If some variables are double precision and others single precision, make sure that you coerce the storage modes appropriately.

Tips

[*]

Troubleshooting

[*]

Most errors are likely to be due to lax attention to storage aspects. So the first question you should ask yourself is whether you've coerced the storage modes of your arguments correctly.

Frequently Asked Questions

  1. I get an error message such as: 
    Error in .C("S_QPE_shobjlist_load",: Can't load (dlopen) library ./libfoo.so:
        21859:/usr/local/lib/S-PLUS/cmd/Sqpe: rld: Fatal Error: unresolvable
        symbol in ./libfoo.so: bar
    This is due to the fact that your program references other functions or routines that could not be found. Make sure that you have all of them available. In the above example, the function bar could not be located. If the file bar.c contains that function, compile it and put both foo.o and bar.o into the library. The short way to do this is of course to use: 
    rgmiller> Splus SHLIB -o libfoo.so foo.c bar.c

  2. I get an error message such as: Error in .C("S_QPE_shobjlist_load",:...), even when I see that there is a file libfoo.so in my directory.

    Use the full pathname of the file such as dyn.load.shared("./libfoo.so"). One can set environment variables like LD_LIBRARY_PATH but I have good reasons to maintain that the former is the easiest and cleanest solution.

  3. A friend gave me his Fortran code for doing blah. I want to call the routine from Splus. If I do the straightforward thing, I get errors. How should I proceed?

    Follow these steps.

    1. Is the code giving you a function or a subroutine? If the former, convert it into the latter. This is trivial to do if you know fortran.
    2. Is the routine using print or write statements? If so, comment them out. If the program is small enough, this is usually easy to do. However, such statements might provide important diagnostic information. In other situations, the program might be so large that it might be impossible to comment all of them out safely without clobbering the program. Then you must use the routines DBLEPR, REALPR, or INTPR described in [cite vena:ripl:1994] for printing the output. If the print or write statements provide no useful information that can be obtained otherwise, you can trick Splus by writing some dummy routines. Again, refer to [cite vena:ripl:1994].
  4. I made a change to my file foo.c, recompiled and reloaded the dynamic library using dyn.load.shared and yet my program behaves exactly as if the old version of foo.c is in effect. What gives?

    Read the help page for dyn.load.shared carefully. Use the symbols argument to force dyn.load.shared to read in the redefined routines.

    The dynamic loading mechanism loads the definition of a symbol by first checking if it is already defined. If so, no loading of the symbol is done. Thus, if you repeatedly load a shared library using the bare dyn.load.shared function, you'll be effectively doing nothing. The symbols argument forces reloading of the symbol definition. It is therefore prudent to use the symbols argument until you have completely debugged your dynamically loaded routines.

Remarks

[*]

In my opinion, the whole issue of dynamic loading is made unnecessarily complicated in Splus by having functions like dyn.load, dyn.load2, and dyn.load.shared. There should be one function called dyn.load that does the appropriate thing for the platform. For example, Lisp-Stat[cite tier:1990], which borrowed some of the dynamic concepts ideas from S has done this very thing: just one function called dyn-load does different things on different platforms. I hope this gets cleaned up soon.

Dynamically loading C routines is really no different, except that the symbols can be specified to the dyn.load.shared function via symbol.C instead of symbol.F.

Indices

[*]

Code Chunks

[*] This index is generated automatically. The numeral is that of the first definition of the chunk.

Index of Identifiers

[*] Here is a list of the identifiers used, and where they appear. Underlined entries indicate the place of definition. This index is generated automatically.

References

[1] Luke Jon Tierney. Lisp-Stat. An object-Oriented Environment for Statistical Computing and Dynamic Graphics. John Wiley &Sons, 1990.

[2] W. N. Venables and Brian D. Ripley. Moden Applied Statistics with S-Plus. Springer Verlag, 1994.