[toggle ads]

Odi's astoundingly incomplete notes

New Entries

qemu: SHA1 hash support is required for websockets

That error message from qemu wants to tell you that you need to compile it with gnutls support enabled. Set the gnutls USE flag on Gentoo.

posted on 2016-02-05 18:50 CET in Code | 0 comments | permalink

Allowing normal users to restart Apache using polkit

Allow an ordinary unprivileged user to restart a system service like Apache is a frequent requirement in operations. I enounter this on a daily basis when the customer (owner of the machine) asks me (manufacturer of the application) to configure Apache for our application or to troubleshoot an issue. Of course they don't just want to give me complete root access. So what do they do? Usually it's just sudo /etc/init.d/httpd restart.

I am very uneasy with passing user controlled arguments to shell scripts. The whole command should be wrapped into a script that does its job without any arguments.

And in a second step you could do it also via polkit. As that's a bit verbose, here is how.

1. Create the program to run. Here we simply execute the command to restart Apache.
exec /etc/init.d/httpd restart

2. Create the policy for pkexec. Customize the action ID (freely chose the part after pkexec).
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE policyconfig PUBLIC
 "-//freedesktop//DTD PolicyKit Policy Configuration 1.0//EN"

  <action id="org.freedesktop.policykit.pkexec.httpd-restart">
    <description>Restart Apache</description>
    <message>Authentication is required to restart Apache</message>
    <annotate key="org.freedesktop.policykit.exec.path">/usr/local/sbin/httpd-restart</annotate>

3. Define a rule to which user this applies.
polkit.addRule(function(action, subject) {
               if (action.id == "org.freedesktop.policykit.pkexec.httpd-restart" &&
                   subject.user == "john.doe") {
                   return polkit.Result.YES;

4. Now user john.doe can safely execute this without entering a password:
pkexec /usr/local/sbin/httpd-restart
Security notes:
pkexec is an suid binary. It performs checks via polkit and then executes the passed command. It essentially works the same as sudo (which is also an suid binary). The same precautions to running a binary as root apply: don't let users execute random binaries as root if those binaries can also do their job as a normal user.

That means: don't run an editor (vim) as root just to edit a file that happens to be owned by root. Editing doesn't require root privileges. Neither does file access. Let them run their favourite editor themselves and give them access to the file with permissions/ACLs. See man setfacl.

posted on 2016-01-24 14:49 CET in Code | 0 comments | permalink

sudo is a complete failure

Today it was about the 10th time that a customer gave me permissions to edit a config file by allowing me to sudo vi. If you don't see the problem, then read why this gives you full root access.

$ sudo -l
Matching Defaults entries for xxx on this host:

User xxx may run the following commands on this host:
    (root) NOPASSWD: /usr/bin/vi /etc/apache2/*, (root) /usr/bin/vi /etc/apache2/conf.d/*, (root) /usr/bin/less
    /var/log/apache2/*, (root) /etc/init.d/httpd
OMG wildcards! And I also control PATH! There are so many possibilities to get full root access from this misconfiguration, I don't even bother to list them here.
Of course there is also the lazy admin that doesn't care at all and tells you to use sudo su, which not only gives you a full root shell in the first place but also leaves you with a polluted environment.

Apparently many system admins think that sudo has something to do with security. And obviously they have not the slightest concept of what it actually does. What it actually does is: it is an binary with the suid bit set (always runs as root), that simply checks rules in the /etc/sudoers and then executes the passed command verbatim as a root process.

As we all know: executing random programs as root is a very bad idea in the first place. Doing that through sudo isn't any safer.

Apple and Ubuntu use it to obfuscate the fact that they basically run everything with the possibility to obtain root privileges by merely knowing the current user's password (if any at all).

sudo should be removed from computers. It is security theater and does more harm than good by tricking admins into running completely insecure software as root.

If you need to give people access to config files, then use file system permissions (hello we have had ACLs for many years now!). Yes I know by letting john.doe edit Apache config he can also easily get root. But that's not the point here. The point is abuse of sudo to work around file system permissions.
# allow john.doe to edit a single file
setfacl -m 'u:john.doe:rw' /etc/apache2/httpd.conf
# allow john.doe to edit all apache config
setfacl -m 'u:john.doe:rwx' /etc/apache2/conf.d
setfacl -m 'u:john.doe:rw' /etc/apache2/conf.d/*.conf
If you are scared to set the suid bit on the executable itself, then you shouldn't run it via sudo either.

posted on 2016-01-21 15:42 CET in Code | 0 comments | permalink

Web Cruft

Two good articles:
posted on 2016-01-12 17:48 CET in Code | 0 comments | permalink

Rename an Oracle instance

Here is how to rename an Oracle instance.

Also change the instance name in /etc/oratab!

If you also change the IP/hostname, don't forget to:
posted on 2015-08-05 12:40 CEST in Code | 0 comments | permalink

Shrink your Oracle database

One thing, Oracle's Enterprise Manager Webapp miserably fails at is executing shrink recommendations that are given by its Automatic Segment Advisor. Here is a little procedure and log table for your SYS schema that does that well.

It shrinks tables and indexes when it saves more than 100MB. It works correctly for tables with function indexes and LOBs. It works even for a large number of objects. A failure in one object doesn't prevent work on other objects. It stops during office hours. And you get a nice log of what happened. Run it from a periodic job in your development instance and forget about it.

Interesting that 50 lines of code can do better than the EM.
  DT    DATE  DEFAULT current_date  NOT NULL,
  TEXT  VARCHAR2(4000)

 v_err VARCHAR2(4000);
 v_memento VARCHAR2(4000);
delete from shrink_log where dt < current_date - 90;
insert into shrink_log (text) values ('starting');

for r in (SELECT segment_owner||'.'||segment_name as t, segment_owner as owner, segment_name as table_name, segment_type
            FROM TABLE (DBMS_SPACE.asa_recommendations ('TRUE', 'FALSE', 'FALSE'))
            where segment_type IN ('TABLE', 'INDEX')
            and reclaimable_space > 100E6
            order by segment_type desc, reclaimable_space asc) loop
  -- stop during the day
  if ((to_number(to_char(current_date, 'HH24')) >= 5) AND (to_number(to_char(current_date, 'HH24')) < 21)) then
    insert into shrink_log (text) values ('exiting');
  end if;
    v_memento := NULL;
    if r.segment_type='TABLE' then
      for i in (SELECT di.owner||'.'||di.index_name as idx, ie.column_expression FROM dba_indexes di, DBA_IND_EXPRESSIONS ie
                 WHERE di.index_type LIKE 'FUNCTION-BASED%'
                   and di.owner=r.owner and di.table_name=r.table_name
                   and ie.index_owner=di.owner
                   and ie.index_name=di.index_name) loop
        v_memento := v_memento || 'execute immediate ''create index '|| i.idx ||' on '|| r.t ||' ('|| i.column_expression ||')'';';
        execute immediate 'drop index '|| i.idx;
      end loop;
      execute immediate  'alter table '|| r.t ||' enable row movement';
      execute immediate  'alter table '|| r.t ||' shrink space cascade';
      execute immediate  'alter table '|| r.t ||' disable row movement';
      execute immediate  'alter index '|| r.t ||' shrink space compact cascade';
    end if;
    if v_memento is not null then
      execute immediate 'begin '|| v_memento ||' end';
    end if;
    insert into shrink_log (text) values (r.t||': OK');
    when others then
      v_err := substr(SQLERRM, 1, 3900);
      insert into shrink_log (text) values (r.t||': '||v_Err);
end loop;
insert into shrink_log (text) values ('complete');

posted on 2015-05-19 15:35 CEST in Code | 1 comments | permalink
Nice. Very Nice.

git clone --reference with Cygwin

If you use git clone with the --reference option under Cygwin, the resulting repository will not work with native apps like Eclipse. The reason for that is that the .git/objects/info/alternates file contains an absolute Cygwin path (/cygdrive/c/...) which native apps can not resolve.

Clever solution: edit the file and use a relative path (forward slashes). The path must be relative to .git/objects, so use at least three levels up (../../../) as a prefix. Now the repository works with native apps and from the Cygwin command line.

posted on 2015-05-04 16:58 CEST in Code | 0 comments | permalink

Crashes after updating microcode-data

Today Gentoo released the Intel Microcode update: sys-apps/microcode-data-20150121
After installation immediately a lot of applications crashed with the following log:
kernel: traps: NetworkManager[1308] trap invalid opcode ip:7f8bba0b093a sp:7fffd40a22e8 error:0 in libpthread-2.20.so[7f8bba0a5000+16000]

Some research revealed that this machine has an Intel Haswell CPU with buggy TSX. And the microcode update fixes this by effectively disabling TSX. This in itself is not a problem yet.

The most popular code that uses TSX is libpthread of glibc. And I had used an too old gcc version to compile glibc, with -march=native in CFLAGS. Unfortunately this gcc version enables TSX (hle, rtm) with the native optimizations enabled. So everything crashed at once.

The fix was to simply recompile glibc with gcc-4.8.4:
# gcc-config -l
 [1] x86_64-pc-linux-gnu-4.8.4 *
# emerge -1 glibc
You can check if gcc disables hle and rtm on your Haswell CPU correctly:
$ gcc -march=native -E -v - </dev/null 2>&1 | sed -n 's/.* -v - //p'
-march=core-avx2 -mcx16 -msahf -mmovbe -maes -mpclmul -mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mbmi2 -mno-tbm -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle -mrdrnd -mf16c -mfsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mxsave -mxsaveopt --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=6144 -mtune=core-avx2 -fstack-protector

posted on 2015-04-15 14:30 CEST in Code | 0 comments | permalink

Do not use Xerces, Xalan any more!

For years Java developers were used to require Apache Xerces and Xalan for their XML stuff. But those times are over. All this stuff is no longer necessary in modern JDKs (since at least 1.6 basically) and it does more harm than good to have these jars in your classpath or endorsed libs!

The problem is that the Internet is littered with references to Xerces and that even their Website is totally misleading. If you read that stuff you would believe that this is actually usefull code. But it isn't any more!

The first sentence on the website is "Welcome to the future!". But that's from at least 2010! The classes in xml-apis.jar are from 2009! All the code is build for JDK 1.3! Come on! Anybody who is still running 1.3 already has a lot of problems with the memory model / thread synchronization, let alone security issues! xml-apis contains ancient versions of JDK standard classes. For instance XMLStreamException doesn't even initialize the Exception's cause field, getting you nothing but incomplete stack traces.

I really wonder why this project can't get around to admit that it is effectively dead. They should really put a big fat warning on their website:


But no! All that 20 year old documentation still tells people to put that crap into the endorsed folder of the JDK where it will turn a perfectly good XML infrastructure into a jurassic version of itself.

These people don't realize how much worldwide developer time they are wasting by keeping this outdated website up!

Please also note that Xerces has security issues when exposed unconfigured to the Internet (webservices!), whereas JDK's built-in JAXP has sane defaults.

If your project still has Xerces, you should migrate away from it now. Chances are that you hard-coded some stuff on top of Xerces (like validation, feature strings, parser options, etc.). Use the modern JAXP equivalents!
posted on 2015-03-09 11:55 CET in Code | 3 comments | permalink
The "welcome to the future" statement at https://xerces.apache.org/xerces2-j/ is surprising indeed. It might make sense to contact those projects directly with your suggestions, if you haven't done that yet.
Already done: https://issues.apache.org/jira/browse/XERCESJ-1657

Guess what: nobody has ever answered.

This project is completely abandoned. Everybody has run away.
Sometimes I feel the same way about all these logging libraries out there, such as log4j, logback, slf4j.

JUL is already an extensible framework, instead of providing appenders and handlers, people started to create whole new frameworks. log4j should just have been allowed to die in the same way.

People tout the advantages of these frameworks in terms of isolated advantages (performance, asynchronicity, audit, etc, etc) , without ever mentioning they could have done the same by providing extensions under JUL.

And then there is the matter of none of them ever bothering unifying the configuration aspect, so even if you manage to put in a facade such as SLF4J, then it is still hell to unify the logging from different libraries using different frameworks.

git: Dangerous default settings in Eclipse

When Eclipse imports a GIT repository, it sets some defaults that I can only describe as outright dangerous:

This setting means that each pull operation will fetch changes for all remote branches and merge them. This is totally unintuitive: it modifies stuff outside of the currently check-out branch. And it is the oposite of the command line git behaviour: you need explicitly specify if you wanted to fetch different branches (git fetch origin somebranch:somebranch)

If you happen to work with many work directories (one for each branch) that are based on the same git repository (git-new-workdir), that one click can get you into a lot of trouble if you are not ready to pull everywhere.

posted on 2015-03-03 19:00 CET in Code | 0 comments | permalink