My Tinderbox is working...

Yes, you heard it right. My tinderbox is working. Even now i don't know why i was trying to setup a tinderbox after both Emily and Nags asked me to concentrate on Firefox automation. I felt it was almost done and just few more steps, i will be able to set it up. And that's what finally happened :)

So in this post, i am going to mention about all issues i faced in setting up a tinderbox and steps to fix them.

Note :: If you are ever setting up a tinderbox, be prepared to see huge log files. Like once my compilation ran for few hours and produced a log file of size 15MB. And if you are stuck up, DON'T send the entire file. A tail of that file should do. In fact, you should be in a position to find out from while line the exact error starts.

Yeah before you start reading this
http://sp2hari.blogspot.com/2007/06/conquered-solaris.html and
http://sp2hari.blogspot.com/2007/06/tinderbox.html has instructions about how to setup Tinderbox in Solaris environment.

Ok the first error i got was

Building deps for jsinterp.c
cc -o jsinterp.o -c -DOSTYPE=\"SunOS5\" -DOSARCH=SunOS -DEXPORT_JS_API -DJS_USE_SAFE_ARENA -I../../dist/include -I../../dist/include/js -I../../dist/include/nspr -DMOZ_PNG_READ -DMOZ_PNG_WRITE -I../../dist/sdk/include -I. -KPIC -xlibmil -xstrconst -xbuiltin=%all -mt -DNDEBUG -DTRIMMED -xO4 -DMOZILLA_VERSION=\"1.9a7pre\" -DMOZILLA_VERSION_U=1.9a7pre -DSOLARIS=1 -DNSCAP_DISABLE_DEBUG_PTR_TYPES=1 -DD_INO=d_ino -DSTDC_HEADERS=1 -DHAVE_ST_BLKSIZE=1 -DHAVE_SIGINFO_T=1 -DHAVE_INT16_T=1 -DHAVE_INT32_T=1 -DHAVE_INT64_T=1 -DHAVE_UINT=1 -DHAVE_UINT_T=1 -DHAVE_UINT16_T=1 -DHAVE_DIRENT_H=1 -DHAVE_SYS_BYTEORDER_H=1 -DHAVE_GETOPT_H=1 -DHAVE_MEMORY_H=1 -DHAVE_UNISTD_H=1 -DHAVE_NL_TYPES_H=1 -DHAVE_MALLOC_H=1 -DHAVE_X11_XKBLIB_H=1 -DHAVE_SYS_STATVFS_H=1 -DHAVE_SYS_STATFS_H=1 -DHAVE_LIBM=1 -DHAVE_LIBDL=1 -DHAVE_LIBSOCKET=1 -DFUNCPROTO=15 -DHAVE_XSHM=1 -D_REENTRANT=1 -DHAVE_RANDOM=1 -DHAVE_STRERROR=1 -DHAVE_LCHOWN=1 -DHAVE_FCHMOD=1 -DHAVE_SNPRINTF=1 -DHAVE_MEMMOVE=1 -DHAVE_RINT=1 -DHAVE_STAT64=1 -DHAVE_LSTAT64=1 -DHAVE_FLOCKFILE=1 -DHAVE_LOCALTIME_R=1 -DHAVE_STRTOK_R=1 -DHAVE_LANGINFO_CODESET=1 -DVA_COPY=va_copy -DHAVE_VA_COPY=1 -DHAVE_I18N_LC_MESSAGES=1 -DMOZ_EMBEDDING_LEVEL_DEFAULT=1 -DMOZ_EMBEDDING_LEVEL_BASIC=1 -DMOZ_EMBEDDING_LEVEL_MINIMAL=1 -DMOZ_PHOENIX=1 -DMOZ_BUILD_APP=browser -DMOZ_XUL_APP=1 -DMOZ_DEFAULT_TOOLKIT=\"cairo-gtk2\" -DMOZ_WIDGET_GTK2=1 -DMOZ_ENABLE_XREMOTE=1 -DMOZ_THEBES=1 -DMOZ_CAIRO_GFX=1 -DMOZ_X11=1 -DMOZ_DISTRIBUTION_ID=\"org.mozilla\" -DMOZ_ENABLE_XFT=1 -DMOZ_ENABLE_PANGO=1 -DMOZ_ENABLE_COREXFONTS=1 -DMOZ_ENABLE_GNOMEUI=1 -DMOZ_EXTRA_X11CONVERTERS=1 -DOJI=1 -DIBMBIDI=1 -DMOZ_VIEW_SOURCE=1 -DACCESSIBILITY=1 -DMOZ_XPINSTALL=1 -DMOZ_JSLOADER=1 -DNS_PRINTING=1 -DNS_PRINT_PREVIEW=1 -DMOZ_NO_XPCOM_OBSOLETE=1 -DMOZ_XTF=1 -DMOZ_MATHML=1 -DMOZ_ENABLE_CANVAS=1 -DMOZ_SVG=1 -DMOZ_SVG_FOREIGNOBJECT=1 -DMOZ_UPDATE_CHANNEL=default -DMOZ_PLACES=1 -DMOZ_PLACES_BOOKMARKS=1 -DMOZ_FEEDS=1 -DMOZ_STORAGE=1 -DMOZ_SAFE_BROWSING=1 -DMOZ_URL_CLASSIFIER=1 -DMOZ_LOGGING=1 -DMOZ_USER_DIR=\".mozilla\" -DMOZ_ENABLE_LIBXUL=1 -DHAVE_STDINT_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_SYS_INT_TYPES_H=1 -DHAVE_UINT64_T=1 -DMOZ_XUL=1 -DMOZ_PROFILELOCKING=1 -DMOZ_RDF=1 -DMOZ_MORKREADER=1 -DMOZ_DLL_SUFFIX=\".so\" -DXP_UNIX=1 -DUNIX_ASYNC_DNS=1 -DJS_THREADSAFE=1 -DMOZ_ACCESSIBILITY_ATK=1 -DATK_MAJOR_VERSION=1 -DATK_MINOR_VERSION=12 -DATK_REV_VERSION=3 -DMOZILLA_LOCALE_VERSION=\"1.9a1\" -DMOZILLA_REGION_VERSION=\"1.9a1\" -DMOZILLA_SKIN_VERSION=\"1.8\" -D_MOZILLA_CONFIG_H_ -DMOZILLA_CLIENT jsinterp.c
ube: error: Assert has been violated at '/set/venus/builds.intel-S2/nightly.Thu/intel-S2/lang/ube/graphs/src/scregion.c 305'.
cc: ube failed for jsinterp.c
gmake[4]: *** [jsinterp.o] Error 2

Dave from Sun china asked me use a patch for Sun Studio 11 Compiler C. I don't have the link to the patch now but the patch id is 121016-05. I remember getting it from the sun website. So searching there should give you the link. The above patch was dependent on patch 120759-06. So installed patch 120759-06 first and then install the patch 121016-05.

Steps to add a patch in Solaris are given below. The commands are for patch 121016-05. Change the id according to the patch you are installing.

1. Download 121016-05.zip and extract the contents in your current directory
2. Login as root
3. root@home# patchadd 121016-05
Note :: If the above command doesn't work, then try
root@home# patchadd . 121016-05

The above steps should install the patch in your system. To check whether the patch is installed properly, check the folder /var/sadm/patch/. You should see seperate directories for each patch you have installed. In my system here, the /var/sdm/patch is like

# cd /var/sadm/patch/
# ls
120759-06 121016-05

That fixed the above problem :).

The next error i got was

jsinterp.c
ir2hf: error: Ran out of memory
cc: ir2hf failed for jsinterp.c

Now this is a strange problem and which can be fixed without much issue. You need more RAM. My system had 768MB (512+256) RAM, but you need 1GB RAM to fix this issue. So next day i got 1GB RAM and never got this error back.

After that, i got many compilation errors. This happened because of some issue in the mozilla code. This generally never happens. I got errors in lines having inline. Anyway now i am not getting the error anymore. So i think the mozilla code is fixed now. :)

Finally i got an error saying,

gmake[3]: Leaving directory `/export/home/hari/tinderbox

/SunOS_5.11_Depend/mozilla'
gmake[2]: Leaving directory `/export/home/hari/tinderbox/SunOS_5.11_Depend/mozilla'
make[1]: Leaving directory `/export/home/hari/tinderbox/SunOS_5.11_Depend/mozilla'
firefox-bin built successfully.
firefox-bin built successfully.
firefox-bin binary exists, build successful.
Unable to find path to Talkback client. Proceeding as if Talkback was not installed.


Running regxpcom test ...
Timeout = 120 seconds.
Begin: Fri Aug 24 08:08:37 2007
cmd = /export/home/hari/tinderbox/SunOS_5.11_Depend/mozilla//dist/bin/firefox-bin -register
End: Fri Aug 24 08:08:38 2007
----------- Output from regxpcom -------------
----------- End Output from regxpcom ---------
regxpcom: test failed
No profile found, creating profile.
Begin: Fri Aug 24 08:08:38 2007
cmd = /export/home/hari/tinderbox/SunOS_5.11_Depend/mozilla//dist/bin/firefox-bin -CreateProfile default
End: Fri Aug 24 08:08:39 2007
----------- Output from Profile Creation -------------
Xlib: connection to ":0.0" refused by server
Xlib: No protocol specified


(firefox-bin:6432): Gtk-WARNING **: cannot open display:
----------- End Output from Profile Creation ---------
ERROR: profile /export/home/hari/tinderbox/SunOS_5.11_Depend/.mozilla/firefox/ does not exist
no pref file found

The solution for the above bug was to allow connections to X server. This is done by executing the following command before starting a tinderbox compilation. Remember the user who is running the tinderbox should execute the command.

hari@home:~$/usr/openwin/bin/xhost +
access control disabled, clients can connect from any host

Few other things to note while trying to compile firefox under tinderbox are
1. Never ever run two instances of tinderbox at the same time. Make sure the first one is killed before starting a new one.
2. Delete the SunOS_5.11_Depend folder when you are facing any compilation errors.
3. Use -XO3 for CFLAGS and CXXFLAGS
4. Make sure your mozconfig file has these lines
ac_add_options --disable-freetype2
ac_add_options --disable-debug
ac_add_options --disable-tests

Tinderbox ....

You never know when things will work and but if you believe that they will work ,they will. Ha, finally my own quote :P .

The Solaris download which i asked my friend to do failed :-( . Crap . I waited for that for more than 2 days and finally it failed. Now i can't to afford to wait more. I have to proceed with Tinderbox, that too with real pace. With Emily's trip to GUADEC and college reopening soon, i really have to do some magic.

Emily wanted me to update from Solaris Nevada snv_55 to Solaris Nevada snv_64. Even i wanted to see what's new in next version but looks like fate was against me.

As such i had to reinstall my Solaris Nevada snv_55 because while installing i made a stupid mistake of selecting the default options. If you are ever planning to install Solaris, never ever do that. Do choose the custom install . After some two steps it will ask you which hard disk to choose and after that you can edit the partition size. If you fail to do this, Solaris by default allocates only required space to / and rest all to /export/home. For me it ended up having only 400MB free space in / while my /export/home had around 15GB free space :-(. I never wanted to try gparted and other things. A reinstall will fix all the problems and it did.

So finally my partition size fixed and i am ready to start Tinderbox, when the postman gave a parcel for me. It was Solaris Nevada snv_64 from SUN, which i had registered few days back. I didn't know whether to cry or laugh. Felt like banging my head on the monitor. Murphy is GOD . :-)

So made another install of Solaris Nevada snv_64 and finally was ready to start with Tinderbox. Now i had Solaris Nevada snv_64 and all the softwares required. A new and nice thing about this version is a Shutdown button in the start menu. I really missed that in the previous version.

Another small thing about networking in Solaris. If you want your Solaris box to work with dns entries provided, then you should edit the /etc/nsswitch.conf as follows .

A line in /etc/nsswitch.conf will be as follows
hosts: files
Change that line to
hosts: files dns
This will make sure that first /etc/hosts and then dns entires are looked during networking.

The mail from Emily had the following details with this attachment.

Pre-steps
1. Install the latest Solaris Express(s11)
2. Install SunOne Studio compiler
3. Install JDS CBE

1. Set up environment var. (See myenv.sh attached)
2. Check out tinderbox client from community
$ . /path/to/myenv.sh
$ cvs co -d tinderbox mozilla/tools/tinderbox
3. Some modification to be done in the tinderbox source code
1) post-mozilla.pl
$ cd tinderbox; mv post-mozilla.pl post-mozilla.pl.bak
2) tinder-confg.pl (modify the one attached and use it)
$BuildAdministrator = "%YOUR_ALIAS"
$CVS = 'runsocks cvs -q': (only necessary if you use proxy to access Internet)
3) mozconfig (use the one attached)

4, this step is necessary only if you access Internet via proxy
1) build-seamonkey-util.pl(see the patch file build-seamonkey-util.pl.diff)
2) set http_proxy and ftp_proxy in $HOME/.wgetrc since tinderbox client uses wget to send test result to server

5, let's go -- start up your tinderbox client
$ . /path/to/myenv.sh
$ cd /path/to/tinderbox && ./build-firefox.pl --depend -t MozillaTest > ~/build-firefox.log 2>&1 &

So things were really were clear in her mail.
As of now, i was in pre-steps with just Solaris 11 and neither a Solaris studio nor a CBE . So i started with Solaris studio.

The nice thing about Solaris is that the installing a new software is a matter of few clicks. Almost all the installers in Solaris works right out of the box. So with a few seconds i had Solaris studio installed . I had to change my PATH variables like

export PATH=/opt/SUNWspro/bin:$PATH
export MANPATH=/opt/SUNWspro/man:$MANPATH

So after this, you will have to run sunstudio, but i got the following error saying "java 2 SDK not found"
Omg, so now i had to install j2sdk. Downloaded the file "j2sdk-1_4_2_15-solaris-i586.sh" and when i ran that shell script, it created a folder in the current directory with quite a few files.

No i had no clue what to do that. Finally pinging a few of my friends, came to know that i had to set JAVA_HOME and change PATH variable again. I copied the folder that shell script created (j2sdk1.4.2_15) to /opt and had to modify the environment variables as shown below.

export JAVA_HOME=/opt/j2sdk1.4.2_15
export PATH=/opt/j2sdk1.4.2_15/bin:$PATH

Now if you run sunstudio, things work fine :-) . So now 2/3 pre-steps done. The only step remaining is Common Build Environment (CBE).

Now this again wasn't that difficult. The only catch is to give the proper C compiler. As such, i was supposed to use the Solaris compiler that comes along with Sun studio and not gcc. So first time i made a mistake of giving the path to gcc but next time did that fine.

Phew, all pre-steps done. Now to setup tinderbox. All the steps mentioned there worked right out of box. In step 2, it needed the environment variable $CVSROOT. As such $CVS_ROOT was configured in Emily's script. So just changed that to $CVSROOT and things worked fine :-)

Now finally i ran tinderbox and i am not quite sure about the ouput it produced . It surely generates a lot of output . I had no clue what the output meant and what i am supposed to do with that.

The output i got was like

Name "TreeSpecific::clobber_target" used only once: possible typo at ./build-firefox.pl line 33.
Name "TreeSpecific::build_target" used only once: possible typo at ./build-firefox.pl line 31.
Name "TreeSpecific::checkout_clobber_target" used only once: possible typo at ./build-firefox.pl line 13.
Name "TreeSpecific::extrafiles" used only once: possible typo at ./build-firefox.pl line 34.
Possible unintended interpolation of @gmail in string at (eval 318) line 23.
Starting dir is : /root/tinderbox/SunOS_5.11_Depend

tinderbox: tree: MozillaTest
tinderbox: builddate: 1183129260
tinderbox: status: building
tinderbox: build: SunOS/i386 5.11 home Depend trunk
tinderbox: errorparser: unix
tinderbox: buildfamily: unix
tinderbox: version: $Revision: 1.1 $
tinderbox: END

Opening SunOS_5.11_Depend.log
current dir is -- home:/root/tinderbox/SunOS_5.11_Depend
Build Administrator is sp2hari.com
uname -a = SunOS home 5.11 snv_64a i86pc i386 i86pc
AB_CARDCATALOG=/usr/dt/share/answerbooks/C/ab_cardcatalog
ADDON_PATH=/root/tinderbox/SunOS_5.11_Depend/mozilla//dist/bin:
COLORTERM=gnome-terminal
CVSROOT=:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot
CVS_ROOT=:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot
DBUS_SESSION_BUS_ADDRESS=unix:path=/tmp/dbus-zAaS6sa66Z,guid=9c8119a191db027be12633004684e7cc
DESKTOP_STARTUP_ID=
DISPLAY=:0.0
DTAPPSEARCHPATH=/root/.dt/appmanager:/usr/dt/appconfig/appmanager/%L:/usr/dt/appconfig/appmanager/C
DTDATABASESEARCHPATH=/root/.dt/types,/usr/dt/appconfig/types/%L,/usr/dt/appconfig/types/C
DTDEVROOT=
DTHELPSEARCHPATH=/root/.dt/help/root-home-0/%H:/root/.dt/help/root-home-0/%H.sdl:/root/.dt/help/root-home-0/%H.hv:/root/.dt/help/%H:/root/.dt/help/%H.sdl:/root/.dt/help/%H.hv:/usr/dt/appconfig/help/%L/%H:/usr/dt/appconfig/help/%L/%H.sdl:/usr/dt/appconfig/help/%L/%H.hv:/usr/dt/appconfig/help/C/%H:/usr/dt/appconfig/help/C/%H.sdl:/usr/dt/appconfig/help/C/%H.hv
DTSCREENSAVERLIST=StartDtscreenSwarm StartDtscreenQix StartDtscreenFlame StartDtscreenHop StartDtscreenImage StartDtscreenLife StartDtscreenRotor StartDtscreenPyro StartDtscreenWorm StartDtscreenBlank
DTSOURCEPROFILE=true
DTUSERSESSION=root-home-0
DTXSERVERLOCATION=local
EDITOR=/usr/dt/bin/dtpad
GNOME_DESKTOP_SESSION_ID=Default
GNOME_KEYRING_SOCKET=/var/tmp/keyring-a1aGab/socket
GTK_RC_FILES=/etc/gtk/gtkrc:/root/.gtkrc-1.2-gnome2
G_BROKEN_FILENAMES=yes
G_FILENAME_ENCODING=@locale,UTF-8
HELPPATH=/usr/openwin/lib/locale:/usr/openwin/lib/help
HOME=/root
JAVA_HOME=/opt/j2sdk1.4.2_15
LANG=C
LC_ALL=C
LC_CTYPE=C
LD_LIBRARY_PATH=/root/tinderbox/SunOS_5.11_Depend/mozilla/dist/bin:
LIBPATH=/root/tinderbox/SunOS_5.11_Depend/mozilla//dist/bin:
LIBRARY_PATH=/root/tinderbox/SunOS_5.11_Depend/mozilla//dist/bin:/root/tinderbox/SunOS_5.11_Depend/mozilla//dist/bin/components:
LOGNAME=root
MAIL=/var/mail/root
MANPATH=/opt/SUNWspro/man:/usr/dt/man:/usr/man:/usr/openwin/share/man
MOZCONFIG=/root/tinderbox/mozconfig
MOZILLA_FIVE_HOME=/root/tinderbox/SunOS_5.11_Depend/mozilla//dist/bin
MOZ_BYPASS_PROFILE_AT_STARTUP=1
MOZ_CO_DATE=06/29/2007 15:01 +0000
MOZ_PACKAGE_MSI=0
MOZ_SYMBOLS_TRANSFER_TYPE=scp
NLSPATH=/usr/dt/lib/nls/msg/%L/%N.cat:/usr/dt/lib/nls/msg/C/%N.cat
NO_EM_RESTART=1
OLDPWD=/root
OPENWINHOME=/usr/openwin
PATH=/opt/csw/bin:/opt/SUNWspro/bin:/opt/SUNWspro/bin:/opt/j2sdk1.4.2_15/bin:/usr/sbin:/usr/bin:/usr/dt/bin:/usr/openwin/bin:/usr/ucb:/root/tinderbox/SunOS_5.11_Depend/mozilla//dist/bin
PWD=/root/tinderbox
SDT_NO_DTDBCACHE=1
SDT_NO_TOOLTALK=1
SESSIONTYPE=altDt
SESSION_MANAGER=local/home:/tmp/.ICE-unix/316,inet6/home:35165,inet/home:60138
SESSION_SVR=home
SHELL=/bin/bash
SHLVL=2
SSH_AGENT_PID=321
SSH_AUTH_SOCK=/tmp/ssh-XXXAaGNa/agent.316
START_SPECKEYSD=no
TERM=xterm
TZ=Asia/Calcutta
USER=root
WINDOWID=41943093
XFILESEARCHPATH=/etc/dt/app-defaults/%L/%N:/etc/dt/app-defaults/C/%N:/usr/dt/app-defaults/%L/%N:/usr/dt/app-defaults/C/%N:/usr/openwin/lib/locale/%L/%T/%N%S:/usr/openwin/lib/%T/%N%S
XMBINDDIR=/usr/dt/lib/bindings
XMICONBMSEARCHPATH=/root/.dt/icons/%B%M.bm:/root/.dt/icons/%B%M.pm:/root/.dt/icons/%B:/usr/dt/appconfig/icons/%L/%B%M.bm:/usr/dt/appconfig/icons/%L/%B%M.pm:/usr/dt/appconfig/icons/%L/%B:/usr/dt/appconfig/icons/C/%B%M.bm:/usr/dt/appconfig/icons/C/%B%M.pm:/usr/dt/appconfig/icons/C/%B
XMICONSEARCHPATH=/root/.dt/icons/%B%M.pm:/root/.dt/icons/%B%M.bm:/root/.dt/icons/%B:/usr/dt/appconfig/icons/%L/%B%M.pm:/usr/dt/appconfig/icons/%L/%B%M.bm:/usr/dt/appconfig/icons/%L/%B:/usr/dt/appconfig/icons/C/%B%M.pm:/usr/dt/appconfig/icons/C/%B%M.bm:/usr/dt/appconfig/icons/C/%B
XPCOM_DEBUG_BREAK=warn
_=./build-firefox.pl
dtstart_sessionlogfile=/dev/null
-->mozconfig<----------------------------------------
# Make flags
mk_add_options MOZ_CO_PROJECT=browser
mk_add_options MOZ_CO_MODULE="mozilla/tools/codesighs"

# Configure flags
ac_add_options --enable-application=browser
ac_add_options --enable-optimize
ac_add_options --enable-codesighs
ac_add_options --enable-crypto
ac_add_options --enable-xft
ac_add_options --enable-svg
ac_add_options --enable-canvas
ac_add_options --disable-freetype2
ac_add_options --disable-debug
ac_add_options --disable-tests
-->end mozconfig<----------------------------------------
Didn't find /root/tinderbox/post-mozilla.pl
===============================
Compiler is -- CC
===============================
Begin: Fri Jun 29 20:32:28 2007
cvs -q checkout -P -D "06/29/2007 15:01 +0000" mozilla/client.mk mozilla/browser/config
End: Fri Jun 29 20:32:31 2007
/root/tinderbox/SunOS_5.11_Depend/mozilla//dist/bin/firefox-bin does not exist.
No binary detected; none deleted.
Begin: Fri Jun 29 20:32:31 2007
make -f client.mk checkout
make: Fatal error in reader: client.mk, line 126: Unexpected end of line seen
End: Fri Jun 29 20:32:32 2007
Error: CVS checkout failed.

The last line is scary :-(. Have to ask about that to emily or nags soon. And oh yeah, now that i have setup a Tinderbox in Solaris, i want to setup one in Linux. Just want to see whether it is so easy in Linux. ;). Now i got my confidence back :). I can now really finish lots in integration of Firefox Test cases into Tinderbox before i go back to college. And now i am no more scared of Solaris :-) .

Have to figure out something about the Tinderbox log.
Bye for now.