Odi's astoundingly incomplete notes

The tar file format produced by GNU tar is slightly incompatible with Unix tar on Solaris. You will notice that when you tar deeply nested directory hierarchies like java class trees or Java API documentation. Then the Unix tar would complain about @LongLink stuff. Solution: use pax instead of tar.

# this creates an archive of a directory
pax -w -f archive.tar directory
# this extracts an archive
pax -r < archive.tar

The only drawback: pax doesn't support compression directly. So an extra gzip/gunzip doesn't do harm.

posted on 2007-12-18 16:25 UTC in Code | 5 comments | permalink

Connecting to a Solaris machine, my Cgywin terminal seems unsupported:

top: no termcap entry for a `cygwin' terminal

Note: Solaris' termcap uses terminfo really. No termcap file is ever really read! I worked around it by tar'ing the terminfo db from Cygwin and uploading it to the home directory of that user. First tar and upload Cygwin's terminfo db:

cd /usr/share/terminfo
tar czf terminfo.tgz *
scp terminfo.tgz user@solaris:
rm terminfo.tgz

Then on the remote machine unpack it to the ~/.terminfo directory:

gunzip terminfo.tgz
mkdir .terminfo
cd .terminfo
tar xf ../terminfo.tar
# fix a symlink
rm terminfo
ln -s /usr/share/lib/terminfo
cd ..
rm terminfo.tar
export TERMINFO=${HOME}/.terminfo

Now edit .bash_profile to set the TERMINFO variable on login:

# teach Solaris what my terminal is
export TERMINFO=${HOME}/.terminfo

Now the only thing missing, is a useable vi configuration. On Solaris this is the really old Unix vi, and not vim. So its config file is ~/.exrc. Here we need key mappings for Home, End, PageUp, PageDown, Delete and maybe even the cursor keys. For every key put a mapping in the file like:
map <Escape sequence> <command>

You can enter the escape sequence by pressing Ctrl-V and then the desired key (like PageDown). If the command is also an escape sequence (like Ctrl-F), then also just press Ctrl-V before hitting it. So you should get something like that:

set showmode
" Home
map ^[[1~ 0
" End
map ^[[4~ $
" PgUp
map ^[[5~ ^B
" PgDown
map ^[[6~ ^F
" Del
map ^[[3~ x
" Down
map ^[[B j
" Up
map ^[[A k
" Left
map ^[[D h
" Right
map ^[[C l

posted on 2007-12-18 13:17 UTC in Code | 3 comments | permalink

Anyone engaging in text rendering must read the article over at anti-grain. That is people using the OpenType library, coding on word processors, XServers, PDF viewers, HTML rendering engines, postscript rasterizers, image editors, vector renderers etc. I think it contains many brilliant insights.

posted on 2007-11-03 13:37 UTC in Code | 0 comments | permalink

Do you know the operating system TempleOS (SparrowOS)? I mean the guy is obviously nuts, but I really think the world needs people like him! So cool.

posted on 2007-10-03 12:40 UTC in Code | 0 comments | permalink

The problem in the Java 1.5 printing API mentioned earlier seems to be even bigger. If documents are printed concurrently to different printers, the API may make concurrent calls to OpenSSL (which is not thread-safe). That in turn causes memory corruption and leads to VM crashes. The crashes may occur immediately on malloc in the OpenSSL code or afterwards in arbitrary places (even interpreted Java code). The only workaround is to synchronize all printing API calls to a global lock. Bug report at Sun.

Stack of crashing thread:

Stack: [0x6c377000,0x6c3f8000),  sp=0x6c3f6f74,  free space=511k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libc.so.6+0x71cee]
C  [libc.so.6+0x710fd]  malloc+0x8d
C  [libcrypto.so.4+0x2b500]
C  [libcrypto.so.4+0x2bb5c]  CRYPTO_malloc+0x4c
C  [libcrypto.so.4+0x7883e]  OBJ_NAME_add+0x5e
C  [libcrypto.so.4+0x7d2ee]  EVP_add_cipher+0x6e
C  [libssl.so.4+0x29283]  SSL_library_init+0x33
C  [libcups.so.2+0x6003]  httpInitialize+0x63
C  [libcups.so.2+0x62d8]  httpConnectEncrypt+0x28
C  [libcups.so.2+0x62a8]  httpConnect+0x38
C  [libmawt.so+0x400f]  Java_sun_print_CUPSPrinter_canConnect+0x4f
j  sun.print.CUPSPrinter.canConnect(Ljava/lang/String;I)Z+0
j  sun.print.CUPSPrinter.isCupsRunning()Z+71
j  sun.print.UnixPrintJob.print(Ljavax/print/Doc;Ljavax/print/attribute/PrintRequestAttributeSet;)V+186

Test code that reproduces the problem:

import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.InputStream;
import java.util.concurrent.atomic.AtomicInteger;

import javax.print.DocFlavor;
import javax.print.DocPrintJob;
import javax.print.PrintService;
import javax.print.PrintServiceLookup;
import javax.print.SimpleDoc;
import javax.print.attribute.DocAttributeSet;
import javax.print.attribute.HashDocAttributeSet;
import javax.print.attribute.HashPrintRequestAttributeSet;
import javax.print.attribute.PrintRequestAttributeSet;
import javax.print.attribute.standard.MediaSizeName;

/**
 * Crashes on Linux when printin a PDF to two different printers.
 * 
 * @author Ortwin Glück
 */
public class Test implements Runnable{
    private static AtomicInteger select = new AtomicInteger();
    private PrintService service1, service2;
    private Object latch = new Object();
    private String file;
    
    public static void main(String[] args) {
        if (args.length != 3) {
            System.out.println("Syntax: Test printer printer file");
            return;
        }
        Test app = new Test(args[0], args[1], args[2]);
        app.run();
    }
    
    public Test(String printer1, String printer2, String file) {
        PrintService[] services = PrintServiceLookup.lookupPrintServices(DocFlavor.INPUT_STREAM.AUTOSENSE, null);
        for (PrintService ps : services) {
            if (printer1.equals(ps.getName())) {
                service1 = ps;
            }
            if (printer2.equals(ps.getName())) {
                service2 = ps;
            }
        }
        if (service1 == null)
            throw new IllegalArgumentException("Unknown printer "+ printer1);
        if (service2 == null)
            throw new IllegalArgumentException("Unknown printer "+ printer2);
        File f = new File(file);
        if (!f.exists())
            throw new IllegalArgumentException("File not found: "+ f.getAbsolutePath());
        if (!f.isFile())
            throw new IllegalArgumentException("Not a file: "+ f.getAbsolutePath());
        this.file = file;
    }

    public void run() {
        int THREADS = 20;
        for (int i=0; i<THREADS; i++) {
            Thread t = new Thread(new Job());
            t.setDaemon(false);
            t.start();
        }
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
        }
        
        synchronized (latch) {
            latch.notifyAll();
        }
        
    }
    
    private class Job implements Runnable {
        public void run() {
            synchronized (latch) {
                try {
                    latch.wait();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
            InputStream stream;
            try {
                stream = new FileInputStream(file);
            } catch(FileNotFoundException e) {
                // does not happen
                throw new RuntimeException(e.getMessage(), e);
            }
            
            DocPrintJob dpj;
            if (select.getAndIncrement() % 2 == 0) {
                dpj = service1.createPrintJob();
            } else {
                dpj = service2.createPrintJob();
            }
            DocAttributeSet docaset = new HashDocAttributeSet();
            docaset.add(MediaSizeName.ISO_A4);
            SimpleDoc simpleDoc = new SimpleDoc(stream, DocFlavor.INPUT_STREAM.AUTOSENSE, docaset);
            PrintRequestAttributeSet aset = new HashPrintRequestAttributeSet();
            aset.add(MediaSizeName.ISO_A4);
            try {
                //synchronized (Test.this) {
                    dpj.print(simpleDoc, aset);
                //}
            } catch(Exception e) {
                System.err.print(e.getMessage());
            }            
        }
        
    }
}

posted on 2007-08-28 11:24 UTC in Code | 0 comments | permalink

I have reported the second bug in Sun's JDK-1.5 this year: PrintServiceLookup.lookupPrintServices is not thread-safe on Linux! Most of the time it crashes the VM in an OpenSSL call race and sometimes it just hangs the threads. Workaround: synchronize the call and turn off automatic polling by setting the system property sun.java2d.print.polling=false. Bug report at Sun.

Stack of hanging threads:

Thread 30235: (state = IN_NATIVE)
- sun.print.CUPSPrinter.canConnect(java.lang.String, int) @bci=0 (Interpreted frame)
- sun.print.CUPSPrinter.isCupsRunning() @bci=71, line=352 (Interpreted frame)
- sun.print.UnixPrintServiceLookup.getPrintServices(javax.print.DocFlavor, javax.print.attribute.AttributeSet) @bci=125, line=351 (Interpreted frame)
- javax.print.PrintServiceLookup.getServices(javax.print.DocFlavor, javax.print.attribute.AttributeSet) @bci=65, line=359 (Interpreted frame)
- javax.print.PrintServiceLookup.lookupPrintServices(javax.print.DocFlavor, javax.print.attribute.AttributeSet) @bci=2, line=105 (Interpreted frame)
- Test$Job.run() @bci=42, line=42 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)
Thread 30220: (state = IN_NATIVE)
- sun.print.CUPSPrinter.canConnect(java.lang.String, int) @bci=0 (Interpreted frame)
- sun.print.CUPSPrinter.isCupsRunning() @bci=71, line=352 (Interpreted frame)
- sun.print.UnixPrintServiceLookup.getPrintServices(javax.print.DocFlavor, javax.print.attribute.AttributeSet) @bci=125, line=351 (Interpreted frame)
- javax.print.PrintServiceLookup.getServices(javax.print.DocFlavor, javax.print.attribute.AttributeSet) @bci=65, line=359 (Interpreted frame)
- javax.print.PrintServiceLookup.lookupPrintServices(javax.print.DocFlavor, javax.print.attribute.AttributeSet) @bci=2, line=105 (Interpreted frame)
- Test$Job.run() @bci=42, line=42 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)

Stack of crash:

C  [libcrypto.so.4+0x73bda]
C  [libcrypto.so.4+0x73968]  lh_insert+0xc8
C  [libcrypto.so.4+0x760cb]
C  [libcrypto.so.4+0x76753]
C  [libcrypto.so.4+0x766bf]  ERR_load_ERR_strings+0x2f
C  [libcrypto.so.4+0x76795]  ERR_load_strings+0x25
C  [libssl.so.4+0x29fa3]  ERR_load_SSL_strings+0x43
C  [libssl.so.4+0x2124e]  SSL_load_error_strings+0x1e
C  [libcups.so.2+0x5ffe]  httpInitialize+0x5e
C  [libcups.so.2+0x62d8]  httpConnectEncrypt+0x28
C  [libcups.so.2+0x62a8]  httpConnect+0x38
C  [libmawt.so+0x400f]  Java_sun_print_CUPSPrinter_canConnect+0x4f
j  sun.print.CUPSPrinter.canConnect(Ljava/lang/String;I)Z+0
j  sun.print.CUPSPrinter.isCupsRunning()Z+71
j  sun.print.UnixPrintServiceLookup.getPrintServices(Ljavax/print/DocFlavor;Ljavax/print/attribute/AttributeSet;)[Ljavax/print/PrintService;+125
j  javax.print.PrintServiceLookup.getServices(Ljavax/print/DocFlavor;Ljavax/print/attribute/AttributeSet;)Ljava/util/ArrayList;+65
j  javax.print.PrintServiceLookup.lookupPrintServices(Ljavax/print/DocFlavor;Ljavax/print/attribute/AttributeSet;)[Ljavax/print/PrintService;+2

You can easily verify this with the following test case:

import javax.print.DocFlavor;
import javax.print.PrintService;
import javax.print.PrintServiceLookup;

/**
 * On JDK-1.5.0_12 on Linux x86 this test case hangs or crashes.
 * 
 * @author Ortwin Glück
 */
public class Test implements Runnable{
    Object latch = new Object();
    
    public static void main(String[] args) {
        Test app = new Test();
        app.run();
    }

    public void run() {
        int THREADS = 200;
        for (int i=0; i<THREADS; i++) {
            Thread t = new Thread(new Job());
            t.setDaemon(false);
            t.start();
        }
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
        }
        
        synchronized (latch) {
            latch.notifyAll();
        }
        
    }
    
    private class Job implements Runnable {

        public void run() {
            synchronized (latch) {
                try {
                    latch.wait();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
            PrintService[] services = PrintServiceLookup.lookupPrintServices(DocFlavor.INPUT_STREAM.AUTOSENSE, null);
            System.out.println(services.length);
        }
        
    }
}

posted on 2007-08-22 10:55 UTC in Code | 0 comments | permalink

I had a strange problem on an NFS mounted share. I am able to access the directory, list file, create files all right. But as soon as I invoke vim on a file the vim process just hangs. You can't kill the process either. It turns out the process waits for the NFS filesystem in some way. The syslog proves this with lines like:

Jul 3 17:21:23 livorno kernel: nfs: server 192.168.4.5 not responding, still trying

Logging in as root and doing umount -f on the share unlocks the process.

By analyzing packet dump of the NFS traffic (tcpdump -s 9000 -w pack host livorno and host firenze and port 2049) in Wireshark I found that lots of fragmented UDP packets were transmitted. On the server side ifconfig reports lots of receive errors. And /proc/net/snmp reports many fragmentation errors, more than /proc/sys/net/ipv4/ipfrag_low_thresh. That leads me to the conclusion that the server's kernel throws away all fragmented UDP packets, thus inhibiting proper NFS communication.

I solved this problem by specifying a rsize and wsize smaller than the MTU (1024 < 1500) in the mount options of the share in /etc/fstab:

192.168.4.5:/data/shared  /data/shared  nfs auto,rw,rsize=1024,wsize=1024

It seems many people have had this problem before on several platforms. But nobody has ever published a solution. Here your are.

posted on 2007-07-03 18:11 UTC in Code | 0 comments | permalink

The anti-aliasing configuration in KDE lets you set the sub-pixel hinting. To get an idea of the effect look at the following screenshots. Each window uses a different hinting style. The more hinting, the sharper will fonts look.



posted on 2007-06-19 21:53 UTC in Code | 2 comments | permalink

telnet towel.blinkenlights.nl

(better over IPv6 if you have)

posted on 2007-06-14 19:25 UTC in Code | 0 comments | permalink

How Java uses the stack is basically undocumented. So we developers are left with finding out ourselves. Today I wrote some interesting test code, that delivers surprising results. For my tests I used Sun JDK 1.5.0_12 under RedHat Enterprise Linux 3.

First I wanted to know the stack size that the VM chooses for threads. I found that the ulimit -s parameter has no direct influence on the stack size chosen by the VM. The default is apparently 512kb, and can be changed freely with the -Xss and -XX:ThreadStackSize parameters. But I could not find a difference in behaviour between those parameters. They appear to do the same thing. Further I found that this Linux machine is able to create new threads at a rate of about 5000 per second.

I performed these tests by creating new threads and setting them asleep in a blocking wait call immediately. I kept creating threads until the VM ran out of memory. With 512k stacks the number of threads was around 3700, for 256k stacks around 7300, for 128k around 13700. That leads to the following memory consumption by the stacks:

 3700 x 512kB = 1850MB
 7300 x 256kB = 1825MB
13700 x 128kB = 1713MB

Of course a 32-bit process is limited to 4GB of address space (minus 1 or 2 GB for the kernel). So it is only natural that the memory is close to these 2GB minus the heap size. (Note that Stack is never allocated from the heap.)

Next I tested how deep I could call into a recursive method until I get stack overflows. I did this by modifying the initial program, so that each thread would recurse into a method until it hit stack overflow, then fall asleep. The thread recorded the maximum recursion depth. That's where I got really weird results. Especially with the server VM the maximum depth varies over a wide range between 1750 and 5700 calls (128k stacks)! That's far from constant, which would have been my first guess. With the client VM the number is generally lower but doesn't vary that much: between 1100 and 1650.
 Also the maximum depth seems to be lower at the beginning of the test and increases toward the end.

If someone wants to repeat my tests, here is the test code that I was using. I am very interested to get further insights on how exactly stack works in Java.

posted on 2007-06-14 18:51 UTC in Code | 6 comments | permalink