The command line

GNU/Linux, web development and some other things

Goodbye VirtualBox, Welcome KVM/virt-manager

In fact I don’t have anything against VirtualBox, either the full version or the OSE version. What is more, it is a wonderful piece of software and it never gave me any serious problem. The interface it is one of the most friendly and easy to use that I have found.
In spite of that, I felt that VirtualBox was more focused to the user having a couple of virtual machines running for short periods of time (note: It never had a problem when I let it run several days continuously) than aimed to the datacenter where there are dozens of vms running uninterrupted.
On the other side, when I wanted to have a 64bit guest (in times of VirtualBox 1.5, version 2 supports 64 bit guests and hardware virtualization without problem) it didn’t have support for it. So, searching I found KVM and, in principle at least, it had the same functionality than VirtualBox. But it had some additional very interesting. For example, the ability to migrate the vm (while running) from one host to another. Or at least so they said. The problem. It didn’t have a decent GUI. All controlled by the command line. Of course I don’t have any problem with that, as my blog title very clearly states. But for some simple activities, the number of parameters to write was very overwhelming and, accustomed to VirtualBox simplicity, it was too much typing.
Well, this guys at Red Hat have created virt-manager. And it really aimed high in order to easy the management of vms. I am glad that they are succeeding in their goal, because virt-manager it is ready to take the datacenters by storm. In fact, Ubuntu 8.10 it is fully committed to the virtualization trend and virt-manager it is the tool they chose.
Well, enough words, I want to say to the VirtualBox team, thank you very much, your software it is marvelous and have showed what it is possible. At the same time I say welcome to KVM/virt-manager that, by the way, it is completely integrated to the Linux kernel and in the long term this will result in a very strong contender in the virtualization arena.

So, I followed the instructions from the debian wiki:

# aptitude install kvm virt-manager

Added myself to the libvirt group:

# adduser miguel libvirt

According to the instructions you must now log out and log in in order to use virt-manager and start creating vms for the dozen. But no, there is something you must do before. You must fix the network for virt-manager. Keep reading.

First stop dnsmasq:

# /etc/init.d/dnsmasq stop

Disable autostart for it editing /etc/default/dnsmasq and changing:

ENABLED=1

to

ENABLED=0

This is because virt-manager and libvirtd will start the dnsmasq with the appropriate configuration and the default config for dnsmasq conflicts with it and prevents the “default” network from virt-manager to start.

Now, log out, log in again, restart the libvirtd (or restart the machine) in order to the changes to take effect.

If everything works ok, you should see something like this in your processlist:

$ ps ax|grep dnsmasq
14482 ?        S      0:00 dnsmasq –keep-in-foreground –strict-order –bind-interfaces –pid-file  –conf-file  –listen-address 192.168.122.1 –except-interface lo –dhcp-leasefile=/var/lib/libvirt/dhcp-default.leases –dhcp-range 192.168.122.2,192.168.122.254

If you see a dnsmasq process with status Z (zombie), then review your configuration and the instructions.

Ok, with dnsmasq working correctly, start virt-manager as your user and try to connect to the localhost machine you’ll see in the virt-manager window. Before creating the first virtual machine, we must start the network. Click the “Details” button, after selecting localhost and you’ll see the “Host Details” dialog box. Select the “Virtual Network” tab. There you’ll see the “default” network in an inactive status (it only appears if you are connected to the machine). Start it. Now you can create your virtual machines without problems.

As you can see, the “default” network isn’t configured to autostart itself. And after a while, you get tired of starting it before running or creating a vm. To enable autostart, run the following in the command line (because I didn’t find a way to do it with the GUI):

$ virsh net-autostart default

From now on, the network will always start with the libvirtd.

Now, grab your prefered OS instalation CDs and proceed to create all the virtual machines you want.

Right now we have only used the functionality to create virtual machines in a analogous way to VirtualBox, but if you check the documentation for KVM and virt-manager, you’ll see that you can have a lot of host machines, remotely managed, where you can create virtual machines on demand. Also, you can monitor the resources consumed; start and stop the machines and to configure some kind of network storage for your virtual machines to use. Very enterprisey isn’t? Check the output of

$ virsh help

to have an idea that will come. 


Goodbye Sun-java5-jdk, Welcome Openjdk-6-jdk

With Debian Lenny, I finally can install openjdk instead of the official java from sun.com.

As root:

# aptitude install openjdk-6-jdk

This uninstalled (I think, because I don’t remember if I first uninstalled the sun-java5-jdk :)) the sun-java5-jdk version and installed the new openjdk-6-jdk.

I updated the alternatives from debian:

# update-java-alternatives -s java-6-openjdk

And finally, to test:

# java -version
java version “1.6.0_0”
OpenJDK  Runtime Environment (build 1.6.0_0-b11)
OpenJDK 64-Bit Server VM (build 1.6.0_0-b11, mixed mode)

Great.

Boda

Al fin le llegó el día a Talina.
Se casa.
Con Markus.
Y habrá pachanga.
Y vestido y perfume.
También abrazos y buenos deseos.
Y cuentan que una luna de miel en Cancún.

Así que en nombre de mis amigos y mi familia, les deseamos a Talina y Markus:

MUCHAS FELICIDADES Y NUESTROS MEJORES DESEOS.

Congratulations Herr Wagner and Frau Eslava.

Oracle Database 10g Administrator Certified Associate

Several weeks have passed since the last post. Well, I have been busy attending an Oracle course kindly paid by my employer.
This monday was the first certification exam and I passed it. So, now I am an

Oracle Certified Associate

Now to update my Curriculum Vitae.

The last week of November I will be attending the second course and in early December the second exam will take place, this time for the Oracle Certified Professional. That will be harder than this one, I am sure. So to practice.

Gemstone/S and FastCGI With Lighttpd

I was reading the post from James Foster about configuring Apache/FastCGI for serving Seaside. I prefer Lighttpd because, as its name implies, it needs less resources to run. And with virtual hosting, as I have my servers, memory it is a big factor.

I followed the steps to install Gemstone/S in my machine. Instead of a hosted server, I used my laptop with my user as server.
After testing that Seaside was running OK in my machine, using:

miguel@laptop:~$ source /opt/gemstone/product/seaside/defSeaside
miguel@laptop:~$ startGemstone
miguel@laptop:~$ startSeaside_Hyper 8080

and navigating to:

http://localhost:8080/seaside

I stoped the Hyper Webserver with Ctrl+C.

Now to install lighttpd in my Debian GNU/Linux 4.0 etch machine. As root:

laptop:~# aptitude install lighttpd lighttpd-doc

and backup the default config file:

laptop:~# cp /etc/lighttpd/lighttpd.conf /etc/lighttpd/lighttpd.conf.bak

then use this configuration for lighttpd.conf:

# Modules activated
server.modules = ( “mod_access”, “mod_alias”, “mod_accesslog”, “mod_fastcgi” )
# Document root, maybe there you have other sites also
server.document-root = “/var/www/”
# logs
server.errorlog = “/var/log/lighttpd/error.log”
accesslog.filename = “/var/log/lighttpd/access.log”
# Welcome file
index-file.names = ( “index.html” )
# Do not server files beginning with
url.access-deny = ( “~”, “.inc” )
# Server config
server.port = 80
server.pid-file = “/var/run/lighttpd.pid”
dir-listing.encoding = “utf-8”
server.dir-listing = “disable”
server.username = “www-data”
server.groupname = “www-data”
# FastCGI
# Debug enabled, disable on production sites
fastcgi.debug = 1
# Do FastCGI for anything with prefix /seaside
# load-balancing with 3 hosts
# check-local disable searching the requested file in the lighttpd document root and
# forward the request to the fastcgi hosts
fastcgi.server = ( “/seaside” => (
    ( “host” => “127.0.0.1”, “port” => 9001, “check-local” => “disable”),
    ( “host” => “127.0.0.1”, “port” => 9002, “check-local” => “disable”),
    ( “host” => “127.0.0.1”, “port” => 9003, “check-local” => “disable”)
  )
)

Instead of starting Seaside over Gemstone/S with startSeaside_Hyper <port>, I used the provided script to start 3 gems for fastcgi as per James’ instructions:

miguel@laptop:~$ runSeasideGems start

This started 3 fastcgi gems, listening in ports 9001, 9002 and 9003.

Then I restarted lighttpd:

laptop:~# /etc/init.d/lighttpd restart
Stopping web server: lighttpd.
Starting web server: lighttpd.

And tested that the default index page for lighttpd (located in /var/www/index.html) was served statically (you can add anything here)  from the server document root (/var/www/), pointing the browser to:

http://localhost/

Finally, I tested that the /seaside part was correctly delegated to Gemstone/S using FastCGI, pointing the browser to:

http://localhost/seaside

This time I get the dispatcher from Seaside, from where you can navigate the links from Seaside without problems.

This way you can use, for example, blueprint for the CSS style of your app, or maybe TinyMCE, for having a full fledged text editor. All you have to do is to put them in a directory inside the document root of lighttpd (/var/www in this example) and in the Seaside application, you reference them with the absolut path to your website, for example:

http://mysite.com/styles/blueprint.css

or

http://mysite.com/tinymce/jscripts/tiny_mce/tiny_mce.js

So, the webserver to serve the static content of your site (faster) and Seaside to serve your webapp.

Enjoy.




Okcoba Fest

Yo, Miguel Cobá, en mi carácter de representante absoluto y vitalicio de mí mismo, con fecha 24 de setiembre del año del señor 2008, siendo las 16:39 horas en el horario local, 21:39 hrs en el horario universal, en la centenaria, hermosa y ahora muy atribulada Ciudad de México

DECLARO

oficialmente iniciados los festejos conmemorativos del natalicio de nuestro humilde, sonriente, amigable, entrañable, guapo, inteligente, trabajador y muy modesto representado.
En este tenor

ORDENO

que, a partir de las 23:59 minutos en tiempo local y durante un periodo no menor a 3 días y no mayor a lo que sea saludable para el buen mantenimiento de la salud del hígado humano normal de una persona adulta de 29 años, deberá festejarse, en cualquier lugar donde se encuentre uno, a la hora en que así sea conveniente, solo o en compañía, de manera preferente con el interesado, con toda suerte de sustancias y actividades tradicionalmente usadas al efecto en ocasiones de similar trascendencia, y apegados a la tradición, tanto mexicana como ingenieríl, incluyendo sin limitar:
  • derivados etílicos de la uva, la cebada, el agave azul, el arroz y las frutas dulces, así como bebidas dulces, derivadas de frutas o carbonatadas que produzcan una combinación agradable al gusto humano
  • los derivados lácteos en sus diversas variedades generalmente conocidas como quesos, preferentemente en sus presentaciones edam, de cabra, manchego y con ceniza
  • abrazos, besos (preferentemente de personas del sexo femenino, aunque del pequeño caballero de la mesa redonda se aceptan sin chistar y con emoción), actividades intimas y lúdicas para adultos (con exclusividad total para la filósofa y aspirante a curadora de arte, mejor conocida como Frau Kunst)
  • actividades de mesa incluyendo sin excepción, cartas, dominó, jenga, así como concursos coloquialmente conocidos como fondos y muppets
  • narración de anécdotas típicas, emotivas, avergonzantes, divertidas, por n-ésima ocasión al calor de la velada, así como la tradicional reinterpretación del sentido de la vida, del hombre y sus motivaciones, el espíritu del vino y el secreto escondido en el cuerpo de las mujeres y en la risa de los niños.
  • música al gusto del momento, con especial atención a la preferida por nuestro conmemorado
Sin otro afán que el de agradar y hacer patente nuestro amor y la felicidad que nos da el pretexto para honrar a Baco en su más mundana faceta y a participar de Eros en su más práctica y apapachante expresión, decreto:

QUE INICIE EL “OKCOBA FEST” 2008

PD. La sede principal de los festejos será, como siempre, la residencia oficial de nuestro conmemorado donde se les espera el día sábado 27 a las 19 horas tiempo local y hasta que el cuerpo (o el hígado) aguanten.

PPD. No se aceptan regalos, solamente asistencias, risas, “saluds”, abrazos y llamadas telefónicas (así como sus equivalentes modernos Web 2.0). Abstemios, se les insta a hacer un esfuerzo y dejar atrás sus arcaicas y no sociales prácticas :).

PPPD. Si alguien quiere traer regalos, estos no serán despreciados sino que se aceptarán con el más humilde y sincero agradecimiento.

PPPPD. Pero no son obligatorios los regalos.


Al margen un sello con el escudo nacional que dice: “Patronato Oficial para el Festejo del 29-nario del Natalicio de Miguel Cobá”. Una firma ilegible. Secretario General del Patronato para el festejo de Miguel Cobá.

Extiendo la presente en 2 fojas útiles, en México, Distrito Federal, a los 24 días del mes de setiembre del año de nuestro señor dos mil ocho, a fin de que sea publicada en el diario oficial de la federación, y en los diarios de mayor circulación nacional a fin de que sea de conocimiento general de la ciudadanía.

Miguel Cobá, Presidente del Consejo de Administración del Fideicomiso 2008/09/25MCM/ aprobado por la H. Cámara de Diputados, en su sesión extraordinaria y urgente del 23 de setiembre del año de nuestro señor dos mil ocho, para la creación del Patronato para el Festejo del N-ario del Natalicio de Miguel Cobá. Presente.



F***ing MySQL

One more time, I am having trouble with the MySQL database I administer.
Look, this MySQL database was inherited. It contains ~30 databases, most of them MyISAM type. One of them, the biggest, with more than 30GB of data (150 millions registers). The configuration for MySQL especified a first innodb tablespace file of 20GB and a second file of 10MB, with autoextend. From my.cnf:

innodb_data_home_dir = /data/mysql/data
innodb_data_file_path = ibdata1:20000M;ibdata2:10M:autoextend

I was in the migration from MyISAM to InnoDB, in order to have transaction support. But, as I noted after starting the migration of tables, the innodb files used for storing the mysql tables NEVER shrinks. Never, even if you delete tables, drop tables or drop databases. Always grows but never returns unused space to the operating system. Unbelievable.
According to the bug tracking system for mysql (bugs 1341, 1287) this is a design decision to improve performance: if the space were disposed each time a delete is peformed, it would need to be allocated again when more data were written to the database. So it is only marked as deleted but not returned to the operating system. That is reasonable. In fact it is a good point. But there are times when you just need to free the innodb unused space and IT IS NOT possible, in an automatic or, at least, in a manual way.
Why would you need this. Supose, for example, that I have a 100 GB disk. It is used only for mysql. There is only a database of 40GB in it. So I have ~60GB of space free. One day I need to restore an old backup for this database. I create a new database and restore the old backup inside this database. If I have MySQL configured as all Innodb, the new database would be created in the innodb tablespace, so MySQL will grow the tablespace to say 80GB so the restored backup can be loaded. At this moment, I have 80GB used by the tablespace and only 20GB of free space in the disk. After extracting the data from the backup, I delete the restored database. One would think that the 40GB would be returned to the OS, but NO! The table space size is 80GB although only 40GB are really used, in fact, SHOW TABLE STATUS would show that it is only 50% used. If my original database doesn’t grows to fill the 80GB used by the innodb tablespace or grows very slowly I have actually lost 40GB from my disk and I can not do anything easy to recover that space. In other words, I have lost that space.
Of course there are workarounds for this problem, but they involve dumping and restoring the databases and more important, YOU MUST shutdown the database. That is not always posible. Maybe in a weekend or in the night, but that can be too late if you are running out of disk space.
It is needed a command to force MySQL to free unused space, under the responsability of the DB administrator. At this time, there isn’t any.
So the only way to add more free space to my database was by means of the operating system. My filesystem was like this:

[root@usp02 root]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/cciss/c0d0p3      64G   36G   26G  59% /
/dev/cciss/c0d0p2      99M   11M   83M  12% /boot
/dev/cciss/c0d1p1      67G   62G  2.4G  97% /data
none                  2.9G     0  2.9G   0% /dev/shm

and there were still 3 tables in the migration process. I was feeling real panic. Besides, it is Monday morning. The only feasible moment for shutdown is the next weekend or, in case of extreme urgency, until tonight.
Finally, after several hours searching for a solution for the problem in MySQL, I decided to search ways to free space in the filesystem of the operating system. Happily I found a way to reduce the space reserved for root in a ext2 filesystem from the default 5% of the disk size to 1%. The instructions are here. And I don’t even have to remount the filesystem. The change was applied inmediately.
I just typed:

[root@usp02 root]# tune2fs -m 1 /dev/cciss/c0d1p1
tune2fs 1.32 (09-Nov-2002)
Setting reserved blocks percentage to 1 (177816 blocks)
[root@usp02 root]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/cciss/c0d0p3      64G   36G   26G  59% /
/dev/cciss/c0d0p2      99M   11M   83M  12% /boot
/dev/cciss/c0d1p1      67G   62G  5.1G  93% /data
none                  2.9G     0  2.9G   0% /dev/shm

So I could get 2.7GB that hopefully will last until I can shutdown MySQL, remove the autoextend from the second innodb file and add a third tablespace in another filesystem with enought free space.

One more thing I don’t like about MySQL.

Uff.

Mon Premier Post en Français

Je m’ appelle Miguel. Ma femme s’ appelle Aline. J’ ai un garçon. Il s’ appelle Tristán et il a 1 ans. J’ habite à Mexico, en Mexique. Je suis mexicain. J’ aime Aline et Tristán. Je suis ingenieur et je parle espagnol, anglais, allemand et un peu français.
Comment allez-vous?

Au revoir!

Je Ne Comprends Pas!

El lunes anterior inicié mis clases de francés. No sé muchas cosas aún, sólo números y algunas frases sueltas:

un
deux
trois
quatre
cinq

Je ne comprends pas!

Bonjour, je suis Miguel Cobá.

:)

Stand By

Hoy es domingo, un domingo como tantos, aunque este está nublado y nosotros enfermos. Cansados. Cansados de lo cotidiano y de las fiestas que rompen lo cotidiano. Desveladas, desmañanadas. Cansancio.
Tristán está imparable. Agotador. Ahora que camina es muy difícil aguantarle el paso. Yo tengo gripe, Aline tiene la espalda lastimada. Tristán sigue camine y camine. Hermoso. Y agotador.
En este momento esta en su sillón favorito. Recién acaba de tomar su jugo. Esta “platicando” con su perro de peluche. Aline recostada para aliviar el dolor de su espalda, yo aqui escribiendo.
Estas semanas que han pasado han sido grises. Al menos para mí. No he sentido ganas de hacer nada. Trabajo, curso, transporte público, problemas del trabajo que una vez que cruzas la puerta de salida parecen tan irreales, tan poco merecedoras del estres al que nos sometieron.
Nos mandaron a curso de certificación de una tecnología que no me gusta (es como programar jsp hace 6 años: javascript, html, codigo en una sola página. spaguetti code). Dos semanas de absoluto aburrimiento, desesperación por no aprender algo nuevo, algo INTERESANTE. Además sin internet. Sólo con Windows y sin internet. Es como mi idea del infierno. Al menos estaba instalado el Spider. Pero luego de 30 juegos también es aburrido. Leí 2 libros en ese curso.
Me falta motivación, los libros que comencé a escribir siguen detenidos. Y yo sin ganas de continuarlos. Las tres ideas que tenía para proyectos propios siguen en eso, en ideas.
El tiempo sigue pasando, día tras día, sin detenerse. Sin esperarnos.
Ojalá nos esperara.