Partitions in Solaris work quite a bit differently from partitions in other *nix platforms. First we have the partition. This is what we normally resize using utilities such as gparted. Within the Solaris partition, are slices. These can be thought of as ‘sub partitions’ that contain the filesystems to be mounted in the OS. In fact, Solaris often refers to slices as partitions, which can get a bit confusing at times. First we need to resize the overall Solaris partition, then resize the slice(s) within it.

It’s worth noting that my case is exceptionally simple. I have a single filesystem, on a single slice, on a single partition, on a single disk separate from the root one. I don’t recommend trying any of these steps on a root disk or root partition. Consider yourself warned.

First, we need to find out the device name:

$ df -h /disk1
/dev/dsk/c2t1d0s0      148G   112G    34G    77%    /disk1

Right now, we’re interested not in the slice being mounted but the partition containing the slice. In this case it is c2t1d0p0. I’m sure there’s some other way to figure this out, but in my case I only have one slice and one partition on this particular disk (c2t1d0), so it was easy to figure it out.

Unmount all slices:

# umount /disk1

Dump the disk geometry:

# fdisk -G /dev/rdsk/c2t1d0p0 > /tmp/geom

Then dump the current partition table:

# fdisk -W /tmp/ptbl.tmp /dev/rdsk/c2t1d0p0

I recommend backing up the partition table at this point, in case you need it later.

Also, you will want to record the current slice map. Solaris has a command to help you do this, but it refers to slices as ‘partitions’. These partitions are actually slices in the Solaris partition that (should) encompass the entire disk, and are not to be confused with the disk’s partition table that you will modify using fdisk. You can print the current slices using the format command, selecting your disk, and printing the partition map (‘p’ from the partition menu). It should look something like this:

Part      Tag    Flag     Cylinders         Size            Blocks
  0 unassigned    wm       1 - 19577      149.97GB    (19577/0/0) 314504505
  1 unassigned    wm       0                0         (0/0/0)             0
  2     backup    wu       0 - 19577      149.98GB    (19578/0/0) 314520570
  3 unassigned    wm       0                0         (0/0/0)             0
  4 unassigned    wm       0                0         (0/0/0)             0
  5 unassigned    wm       0                0         (0/0/0)             0
  6 unassigned    wm       0                0         (0/0/0)             0
  7 unassigned    wm       0                0         (0/0/0)             0
  8       boot    wu       0 -     0        7.84MB    (1/0/0)         16065
  9 unassigned    wm       0                0         (0/0/0)             0

Keep this somewhere safe for referencing later.

Edit the ptbl.tmp file you created. At the top, you’ll see your disks geometry. At the bottom is the layout. The ONLY field you are interested in modifying at the bottom is the number of sectors (Numsect).

* /dev/rdsk/c2t1d0p0 default fdisk table
* Dimensions:
*    512 bytes/sector
*     63 sectors/track
*    255 tracks/cylinder
*   22192 cylinders
* Id    Act  Bhead  Bsect  Bcyl    Ehead  Esect  Ecyl    Rsect    Numsect
  191   128  0      1      1       254    63     1023    16065    314552700

In my case, I needed to expand the partition to fill the entire disk. I did this by taking the number of cylinders, subtracting one (since the partition starts at cylinder one and not 0 (Bcyl)), multiplying that number by tracks/cylinder, and multiplying that by sectors/track. So the new Numsect value is: 22191*255*63 = 356498415.

Once that’s been changed, write the new partition table to disk:

# fdisk -S /tmp/geom -F /tmp/ptbl.tmp -I /dev/rdsk/c2t1d0p0

You can run fdisk normally (fdisk /dev/rdsk/c2t1d0p0) in order to verify your changes. Once that is done, run format again, since the solaris partition (slices) map may have been lost. In my case, slice 0 lost it’s information, so I recreated it to start at the same start cylinder as before (1), and take up the rest of the disk. Modify it using ’0′ from the partition menu, and save the new map using the ‘label’ command from within the partition menu.

Part      Tag    Flag     Cylinders         Size            Blocks
  0 unassigned    wm       1 - 22188      169.97GB    (22188/0/0) 356450220
  1 unassigned    wm       0                0         (0/0/0)             0
  2     backup    wu       0 - 22188      169.98GB    (22189/0/0) 356466285
  3 unassigned    wm       0                0         (0/0/0)             0
  4 unassigned    wm       0                0         (0/0/0)             0
  5 unassigned    wm       0                0         (0/0/0)             0
  6 unassigned    wm       0                0         (0/0/0)             0
  7 unassigned    wm       0                0         (0/0/0)             0
  8       boot    wu       0 -     0        7.84MB    (1/0/0)         16065
  9 unassigned    wm       0                0         (0/0/0)             0

The slice should now have been resized. Next is to verify that no data was lost:

# fsck /dev/rdsk/c2t1d0s0

And finally, grow the filesystem to take up the rest of the slice:

# growfs /dev/rdsk/c2t1d0s0

And that’s it! Hopefully this helps people understand Solaris partitioning schemes a little bit better. I learned quite a bit while figuring this out.

cannot submit unicode type file using non-unicode server

Now, this is incredibly nonsensical. I’m integrating an already submitted file from one location to another, and checking it in. Why does Perforce reject it if it’s already been submitted successfully once before?

It turns out that the way perforce handles unicode filetypes changed at some point. Before this change, you could submit unicode files without any problems. After this change, your server needs to be reconfigured as a ‘unicode server’ in order to check in unicode files, even though it’s clearly handling already checked-in files just fine. To me, this seems like an artificial restriction, but I digress.

To resolve this, you just need to reopen the files as text, and resubmit them.

$ p4 reopen -t text 

After that, you can work on the files again as normal. I’m not sure what effect this has on the encoding of the files themselves, so as always, YMMV, but at least you’ll be able to get on with your life.

perfmerge++ warning:
        No mapping for change 293174 in database /x/sourceserver/server/. Rejecting.

Thinking that it was just a dead or missing changeset, I skipped right along and proceeded to put the server into production. Verify came back ok, so I figured everything was fine.

It was not…

It seems that perfmerge has an annoying tendency to quit at the first sign of trouble, then throw a ‘warning’ rather than a full blown error message. In this case, it was about to write out the db.changes* tables, then decided against it. When I brought the server back online, there were file revisions, but no changesets or change descriptions to speak of. This is obviously a huge problem!

I checked both source p4 roots, but couldn’t find the changes in question. Out of desperation, I took checkpoints of both roots, and grepped the entire checkpoint looking for the changeset numbers that were throwing warning. There were two things that I found.

The first thing was that somehow, someone had managed to open files under a pending changeset that didn’t exist. I’m not entirely sure how that happened, but it did. Once that was found, it was a simple matter of reverting all the files opened under that missing changeset.

The second thing I found wrong was that perfmerge apparently had an issue with the way my job names were formatted, which caused any changeset tied to that job to fail to merge. In my case, we don’t make heavy use of jobs, and the one with the issue appeared to be accidentally created. After removing the fix tying the changeset to it and deleting it, the merge proceeded to work fine.

Hopefully this helps some people. I regret that I didn’t save my entire solution somewhere, but the perforce documentation should cover most of what’s required to do the same.

I got stuck in the awkward situation where I had to add a windows machine to our master, but the slave could not even ping it. This could be resolved by updating our routing tables on the slave side, but unfortunately that wasn’t an option here. Instead I chose to try and start the slave using a Windows SSHD solution.

Edit: I’ve found that the data throughput on FreeSSHD using the method I’ve outlined below leaves much to be desired. I’ve since switched over to installing cygwin and using it’s sshd server instead. If you are using this pipe to send or archive files, I highly recommend using cygwin’s sshd server instead, as it’s several orders of magnitude faster. For example, I went from a 45 minute transfer, down to only 1 minute.

First things first, we need to install an ssh server onto the master. This was pretty easy, as I just used . Once installed, you need to make sure to stop the service, and run it by hand as administrator in order to change the settings. It’ll show up in the toolbox on your system, so just right click it and click settings. From there you can set up your new user.

A few gotchas: You’ll want to DISABLE “Use new console engine” on the SSH tab, seen below:

Also, when setting up pubkey authentication, you’ll want to put the key into a file in the “Pubkey folder” that’s defined in the Authentication tab of FreeSSHD. The file needs to have the exact same name as the new user you created. In my case, my user was called “releng” so I made a new text file called “releng” (no extension!) with the contents of my jenkins user’s public key.

Test the connection from the master user by sshing into the slave as the jenkins user:

jenkins@jenkins-master:~$ ssh releng@windows-slave-hostname


Once you’ve verified that passwordless authentication works, now it’s time to set up the slave. Copy the slave.jar from your jenkins install (http://yourjenkinsurl/jnlpJars/slave.jar) onto the slave somewhere. In this example, I’ll assume you placed it into C:\jenkins\.

Give it a shot again:

jenkins@jenkins-master:~$ ssh releng@windows-slave-hostname java -jar C:/jenkins/slave.jar -text

It should return something like:

<===[JENKINS REMOTING CAPACITY]===>rO0ABXNyABpodWRzb24ucmVtb3RpbmcuQ2FwYWJpbGl0eQAAAAAAAAABAgABSgAEbWFza3hwAAAAAAAAAAY=<===[HUDSON TRANSMISSION BEGINS]===>rO0ABQ==

That means it’s working, but there’s still more work to be done to get jenkins talking to it properly. First, set up a new Dumb slave in jenkins like you normally would, and select “Launch slave via execution of command on the Master”. Unfortunately the built-in ssh functionality doesn’t work because of the way FreeSSHD handles stdout and stderr.

Here’s the launch command you need it to run:

bash -c 'ssh releng@windows-slave cmd /c "java -jar C:/jenkins/slave.jar -text 2>C:/jenkins/slave.junk.txt"'

The redirect to slave junk is necessary because the slave.jar outputs informational messages to it’s stderr. Normally these messages are not sent back to the master, but in this case FreeSSHD just pipes stderr into stdout, which would cause problems with the master as it parses the output. I’m not 100% sure if wrapping it in bash is necessary, but it seems to work fine.

Good luck!

#!/usr/bin/python

import marshal
import subprocess, shlex
import re
import os
from datetime import datetime
import time
import smtplib
from email.mime.text import MIMEText

today = int(time.time())

window = 60*60*24*30 #30 days

mailhost = "mailhost"
me = "you@yoursite.com"

maxmails = 10

msg = MIMEText("Perforce server is going down! Lookout!")
msg['Subject'] = "Perforce downtime this weekend"
msg['From'] = me

def getRecentUsersList():
    args = shlex.split("p4 -G users")
    p = subprocess.Popen(args, stdout=subprocess.PIPE)
    userList = []
    while True:
        try:
            user = marshal.load(p.stdout)
            if int(user['Access']) > (today-window):
                userList.append(user)
        except:
            break
    return userList

def getUniqueEmails(userList):
    emails = {}
    for user in userList:
        emails[user['Email']] = 1
    print "got " + str(len(emails.keys())) + " emails"
    return emails.keys()

def sendEmail(emails):
    print "sending email to "+str(emails)
    s = smtplib.SMTP(mailhost)
    s.sendmail(me, emails, msg.as_string())
    s.quit()

def sendEmails(emails):
    emailsubset = []
    for email in emails:
        emailsubset.append(email)
        if len(emailsubset) >= maxmails:
            sendEmail(emailsubset)
            emailsubset = []
    if len(emailsubset) > 0:
        sendEmail(emailsubset)

def main():
    userList = getRecentUsersList()
    emails = getUniqueEmails(userList)
    sendEmails(emails)

main()

$ p4 user -d -f SOMEUSER
User SOMEUSER has file(s) open on 2 client(s) and can't be deleted.

The solutions posted online usually involve either deleting the clients, or ‘masquerading’ as the client host and reverting the files by hand. This is very tedious, so I wrote a python script to do it instead. It’s a bit crude, but all it does is take usernames off the command line, and proceeds to revert all the files that user has open on the server.

#!/usr/bin/python

import marshal
import subprocess, shlex
import re
import os
import sys

def getPerforceResponse(command):
	lst = []
	args = shlex.split(command)
	p = subprocess.Popen(args, stdout=subprocess.PIPE)
	try:
		while 1:
			dictionary = marshal.load(p.stdout)
			lst.append(dictionary)
	except EOFError:
		pass
	return lst

def getOpenedFiles():
	return getPerforceResponse("p4 -G opened -a //...")

def getOpenedFilesForUser(userID, opened):
	openedByUser = []
	for i in opened:
		if i['user'] == userID:
			openedByUser.append(i)
	return openedByUser

def getClient(clientID):
	return getPerforceResponse("p4 -G client -o \"" + clientID + "\"")[0]

def main():
	print "Getting list of open files..."
	opened = getOpenedFiles()
	for user in sys.argv[1:]:
		openedByUser = getOpenedFilesForUser(user, opened)
		print "Found " + str(len(openedByUser)) + " opened files for " + user
		print "reverting..."
		for f in openedByUser:
			print "Reverting " + f['depotFile'] + " on " + f['client']
			client = getClient(f['client'])
			host = client.get('Host', 'p4client')
			response = getPerforceResponse("p4 -G -H " + host + " -c " + f['client'] + " -u " + user + " revert -k \"" + f['depotFile'] +"\"")
			print response

main()

I was nearly about to rebuild my machine when I discovered that it only happened for non-maximized GTK windows. As a last ditch effort, I ran gtk-theme-switch2, and switch my theme. Magically, this fixed the problem!

Part of the issue was that I’m running in Fluxbox, not Gnome or Unity. I think somehow it got the theme messed up during the upgrade, and since neither Gnome nor Unity had the chance to set it to a correct theme, it borked.

I know what you are thinking, “Oh, you probably just gave dd the incorrect parameters and wiped your existing system partition.” Not so, actually. I successfully copied all the data to the right partitions on my disk, and was able to boot into ChromiumOS.

The madness started once ChromiumOS started booting. I was using the latest vanilla version, which worked in live mode fine. When I booted off my main disk, however, it started performing some sort of automated recovery. That’s when my Ubuntu partition got wiped.

I had 3 partitions at the time:

• /dev/sda1 – Ubuntu
• /dev/sda2 – C-STATE
• /dev/sda3 – C-ROOT

Now apparently, the vanilla version of ChromiumOS doesn’t like this setup much. It will always try to use the first available partition as the C-STATE. In this case it was my Ubuntu partition. Since it was the wrong label and filesystem type, ChromiumOS decided to go ahead and reformat it without confirmation.

I also tried having C-ROOT installed on sda1, and it didn’t seem to like that much either. In the end, I had to set up my partitions like this:

• /dev/sda1 – C-STATE
• /dev/sda2 – C-ROOT
• /dev/sda3 – Ubuntu

That seemed to work fine, with one minor exception. There was no initrd.img available on the root for some reason, and specifying the root disk using the label wasn’t working either. The following grub entry was what I ended up using:

menuentry "ChromiumOS" {
	insmod ext2
	set root=(hd0,2)
	linux   /boot/vmlinuz root=/dev/sda2 rw noresume noswap i915.modeset=1 loglevel=1 quiet
}

Note that I had to change the root parameter, as well as completely remove the initrd line. Apparently these two problems are related (a certain something needs to be included in the initrd in order to resolve filesystem labels, and since there is no initrd it doesn’t work).

So that’s it! Word to the wise, it’s always a good idea to do a full backup before trying any of this.

The solution I found is a bit hacky, but it works for me. Basically it boils down to adding nfsvers=3 to the mount parameters. In my case, I was using automount, so I just added the option to every line in my /etc/auto.master. This just forces it to mount using NFSv3, which works because of it’s simplicity, but is possibly more insecure (and not all NFS servers will support it).

If possible, you should really just consider doing things the proper way, and getting idmapd set up on your servers and clients. This is just a quick work-around for when you are backed into a corner like I am.

All of the issues that were opened on the Hudson Jira for the Perforce plugin are still open, but I’m unassigning those that are assigned to me. I’ll continue maintaining the Perforce plugin against Jenkins, so bugs will be tracked there, and any fixes I make will be made in the Jenkins repository. Note that the Jenkins version will still work for Hudson until Oracle/Sonatype and/or the Jenkins Community decide to break cross-compatibility.

I’m not saying the Hudson version is dead, just that it doesn’t have an active maintainer. Someone may take over maintaining it, but as far as I know, only one person has been assigned to working on Hudson and the hundreds of plugins written for it.