Drupal Programmer

Using Wodby (drupal4docker) to create our local development environment.

2019-01-02T19:46:00.004+00:00

Using Wodby https://wodby.com to create a Drupal local development environment using docker.

I have used the official Drupal docker image to create a local dev environment, it certainly does work but does not really tie in with how we currently use and develop on Drupal.

I have checked Wodby drupal4docker and it seems to do what I expect and want the local dev environment.

How to set this up

Obviously there is a documentation on how to set this up https://wodby.com/docs/stacks/drupal/local/ even though the documentation is quite thorough I still struggled a bit so wanted to create a documentation for myself and anyone who happens to stumble on this link.

My setup will consist of a local Drupal 8 code base with custom modules and themes, I save my code base in git including the Drupal code, this may not be really needed in the future as we are moving to using composer but for now I am sticking to keeping the Drupal codebase in my repository.

I also have a separate docker image that I build from this code base which is used for CI and eventually becomes a package which is deployed to production.

If starting a new project use the recommended way to install Drupal with composer like so

composer create-project drupal-composer/drupal-project:8.x-dev drupal8 --stability dev --no-interaction

This will download the Drupal code base, and install dependencies and configure a web root the directory structure looks like this. In the above command “drupal8” is our project root but now where your would traditionally expect the Drupal code base to be, this is under web. For rest of the purpose of this document the project root is “drupal8” and we will use this interchangeably.

Now we will download the latest stable release of docker4drupal which is a collection of files, most importantly the .env file with the parameters which you can define and change to your hearts content and the docker-compose.yml file that defines how these services interact with each other. After copying the files in the project root the folder structure looks like this.

Now we need to delete the docker-compose.override.yml as this is not needed if you use your own codebase and not the Vanilla as it is known. We will also need to create a .env file that allows us to conveniently change the properties without changing the docker compose file, this is much cleaner in my opinion, also the properties may be reused and duplicated hence hence it makes sense to have them in a .env file rather than hardcode in the compose file.

I have created a .env file from https://github.com/wodby/docker4drupal/blob/master/.env we can then remove the properties that we will not use.

I struggled a bit earlier till I realised this would work only If you used the docker for Mac and not docker toolbox, so thats what I did installed docker for Mac enabled the nfs volume section and use the user guided caching to get a better performance for Mac users. So my final docker compose file with things I did not need, removed is as follows (note I have kept portioner and traefik but just don’t show in the docker compose file below for berevity if we decide not to use traefik later we we will to expose the appropriate ports in the compose file.:

We also need to create a docker volume like so:

docker volume create --name=docker-sync

And a host entry for the Drupal site which is 127.0.0.1 drupal.docker.localhost

Also ensure you have docker-sync installed and are running it. If you are just checking the docker4drupal and performance is not that bad, you can just skip the docker-sync parts. The docker-sync uses the docker-sync.yml which is also in the archive we downloaded from docker4drupal earlier. The first time sync will take a little longer but then it should give you a good performance boost. If not using docker-sync the performance will be quite shockingly bad though.

Now we can run docker-compose up -d to create the containers, first time it will be slower as it downloads all the images but following this the Drupal site will be available as http://drupal.docker.localhost:8000, run through the installer, once the site is installed the database files are inside the data folder, docker storage is not persistent unless external volumes are used in this case we have used data folder to store the files. We can also take a dump of the database and keep in our codebase which we can place in the marinade-init folder to pre-populate the site content and configuration.

Add behat in Symfony 4

2018-11-03T11:44:00.001+00:00

To add behat in Symfony 4 i have generally followed this link

https://blog.rafalmuszynski.pl/how-to-configure-behat-with-symfony-4/

However i ran into problems so i thought of sharing this info.

The steps are

run composer req behat/behat --dev

but stumbled on the following error:

~/work/thefootie-behat $ composer req behat/behat --dev

Using version ^3.5 for behat/behat

./composer.json has been updated

Loading composer repositories with package information

Updating dependencies (including require-dev)

Restricting packages listed in "symfony/symfony" to "4.1.*"

Your requirements could not be resolved to an installable set of packages.

Problem 1

- Installation request for behat/behat ^3.5 -> satisfiable by behat/behat[v3.5.0].

- behat/behat v3.5.0 requires symfony/class-loader ~2.1||~3.0 -> satisfiable by symfony/class-loader[3.4.x-dev] but these conflict with your requirements or minimum-stability.

Installation failed, reverting ./composer.json to its original content.

It seems I had to choose to downgrade my minimum stability to dev to install this, as this is dev only packages I don't really mind.

~/work/thefootie-behat $ composer config minimum-stability dev

Following which I did not have any problems with the rest of the instructions.

Installation and adding config in behat.yml was fine, but when i tried to run behat i got the error below:

~/work/thefootie-behat $ ./vendor/behat/behat/bin/behat

PHP Warning: require_once(/Users/devuser/work/thefootie-behat): failed to open stream: Undefined error: 0 in /Users/devuser/work/thefootie-behat/vendor/symfony/dependency-injection/ContainerBuilder.php on line 1097

Warning: require_once(/Users/devuser/work/thefootie-behat): failed to open stream: Undefined error: 0 in /Users/devuser/work/thefootie-behat/vendor/symfony/dependency-injection/ContainerBuilder.php on line 1097

PHP Fatal error: require_once(): Failed opening required '/Users/devuser/work/thefootie-behat/' (include_path='.:/usr/local/Cellar/php/7.2.5/share/php/pear') in /Users/devuser/work/thefootie-behat/vendor/symfony/dependency-injection/ContainerBuilder.php on line 1097

Fatal error: require_once(): Failed opening required '/Users/devuser/work/thefootie-behat/' (include_path='.:/usr/local/Cellar/php/7.2.5/share/php/pear') in /Users/devuser/work/thefootie-behat/vendor/symfony/dependency-injection/ContainerBuilder.php on line 1097

After some googling i came across, a post and downgraded the behat/symfony2-extension to 2.1.5 - this seemed to have resolved the issue.

Turned out we used the behat/symfony2-extension with dev-master stability.

"Downgrading" to 2.1.5 resolved the issue.

After downgrading i can see the demo steps passed!

~/work/thefootie-behat $ ./vendor/behat/behat/bin/behat

Feature:

In order to prove that the Behat Symfony extension is correctly installed

As a user

I want to have a demo scenario

Scenario: It receives a response from Symfony's kernel # features/demo.feature:10

When a demo scenario sends a request to "/" # FeatureContext::aDemoScenarioSendsARequestTo()

Then the response should be received # FeatureContext::theResponseShouldBeReceived()

1 scenario (1 passed)

2 steps (2 passed)

0m0.20s (20.80Mb)

Update Drupal 8 DB to allow Updates to be run again for testing and dev purposes

2018-10-15T15:50:00.000+01:00

When working with Drupal 7 and indeed Drupal 8 we may sometimes need to run updates (hook_update) again. In Drupal 7 updating system table should have been sufficient. In D8 things are a little bit different.

Run a database update as follows:

SELECT collection, name, substring(value, 1, 2500) FROM key_value WHERE collection="system.schema" and name="modulenameinquestion"
- The above will give us the last update hook that was run on this database.
UPDATE key_value SET value = 'i:9999' where collection="system.schema" and name="modulenameinquestion"

Hope this helps.

Use Behat BDD tests with Drupal 7

2018-06-18T16:00:00.001+01:00

Behat is a framework that allows us to write BDD tests, aka given-when-then tests. The language of test specification is quite simple and easy to read and is more business friendly, to know more about BDD you can go here https://inviqa.com/blog/bdd-guide

To use BDD with our drupal 7 site we need to go in the code base, create a folder under sites folder called behat. Installing Behat is done best using composer, so we need to create a composer.json file with the following content

{

"require": {

"drupal/drupal-extension": "~3.0"

"config": {

"bin-dir": "bin/"

}

and save as composer.json

Instructions are from https://behat-drupal-extension.readthedocs.io/en/3.1/localinstall.html adding starting steps or steps to make things clear.

Assume composer is installed system wide (you can find better guides to install composer by searching on google).

composer install

This will install all required dependencies (exactly why composer is such a good idea).

Your console will fill up with a lot of information like this:

Once everything is installed, we are ready to start writing tests - but we need to tell behat where to store our context and features first, so wait ... what are context and features? more about that in the excellent BDD guide i linked above.

Lets just say context is where you have your definitions, transformations and hooks, and features where you define your acceptance criteria or your tests in the given-when-then format.

The first step we need to do is initialise behat, to do that we say

./bin/behat --init

This creates the features and context as discussed above and shows you the following message.

If we have installed behat correctly we should be able to see a lot of context already defined for use in our Drupal projects thanks to Drupal Behat extensions. To see the context and maybe use it for reference later do this

./bin/behat -dl

This should list all the context as below:

There are quite a few context defined, in fact they are enough to get us started and take us much further than that. For eg. we will not write a single custom context but try to reuse existing context and write our features to add content and validate.

Remember this if your BDD tests are not working as expected, this could be more because you have modules that have hooks that modify default drupal behaviour and you may have to write your own context by extending the DrupalContext.

Use ProxySQL to obfuscate data for development.

2018-05-22T16:31:00.002+01:00

Before I begin let me put this out as a protection against the pitch fork guys 🙂 I do not in anyway recommend using live data (albeit sanitised) into dev, we should really have test data and keep updating the test data as application grows and is changed.

But in case when your developers are supporting a really fast moving, highly configurable system there may be a need to use live backups to be regularly sanitised and applied to dev systems. The usual way of doing this is take a backup from production servers, transfer to dev/stage servers, restore to Dev DB and then run some sanitisation scripts.

This seemed to be a bit of a problem as we did not want to have a situation that the mysqldump of live database sits on pre prod servers or may be left there my mistake. Privacy is a big thing today and due to introduction of GDPR we now have a need to eliminate any chances of collecting and storing personal data where we do not need it.

Enter ProxySQL:

I stumbled upon a presentation from Frederic Descamps and Rene Cannao at FOSDEM 2017 where they talk about the data anonymisation for developers https://archive.fosdem.org/2017/schedule/event/proxysqldatamasking/ this is exactly what we needed. I wanted to try that out.

I also found an article from the proxysql blog http://www.proxysql.com/blog/obfuscate-data-from-mysqldump which the same authors show what they did to achieve the data obfuscation. What follows is a step by step guide on how i did this with docker.

First we need our actual database, this is going to represent our live database, for this i am using the official mysql-server container from hub.docker.com and then we need some data; for the data i have used the same database as recommended in the blogpost.

I pull the docker image for Mysql 5.7, expose the 3306 port so I can access the database from my host machine.

docker run --name=mysql1 -p 3306:3306 -e MYSQL_ROOT_PASSWORD=newpassword -e MYSQL_ROOT_HOST=% -d mysql/mysql-server:5.7

In short this will create a mysql 5.7 container with the database running, setting the 'e' params we set the password and MYSQL_ROOT_HOST variable needs to be set as % or the IP address of your docker machine if you need to access this database from outside the container.

For more information on other options and how this works please refer to the https://hub.docker.com/r/mysql/mysql-server/, beware the content will be volatile as we are not mounting any shares or volumes, which should suffice for our tests.

Now i will load this test data which is from https://github.com/datacharmer/test_db.

mysql -u root -h 192.168.99.100 -p < employees.sql

in this case 192.168.99.100 is my docker host, as i am using docker toolbox for mac.

Now for create a proxy sql container.

docker run -d -p 6033:6033 nidash/proxysql:1.4.8

This will create a proxysql server and start it, exposing port 6033

Now proxysql uses 2 ports, 6032 for administering proxysql and 6033 where it acts as a proxy for the mysql servers, ie. all external users connect to the mysql server using port 6033.

docker exec -it mysql -u admin -padmin -h 127.0.0.1 -P6032

After you enter the password, which my default is 'admin' you are presented with a mysql client prompt, beware this is not a true mysql command line interface but a special one for proxy sql administering, this is done so it is easier for DBAs to use a familiar interface.

There is an option to configure ProxySQL using a configuration file, when using with docker the above command can be modified to add a -v (volume mapping option to mount your config file over the default one at /etc/proxysql.cfg)

To allow us to connect to our mysql server through the proxysql proxy, we need a user created on mysql server that allows you to connect, in my case i have created a user as 'msandbox', we also need the actual user to be present on the server in our case we will be using the user 'muser' we need to give appropriate permissions to allow it to connect as well, for the proxy sql side of things we need to add the configuration in as follows:

Setup backend:

INSERT INTO mysql_servers(hostgroup_id,hostname,port) VALUES (1,'192.168.99.100',3306);

Configure monitoring:

UPDATE global_variables SET variable_value='msandbox' WHERE variable_name='mysql-monitor_username';

UPDATE global_variables SET variable_value='msandbox' WHERE variable_name='mysql-monitor_password';

Load the variables to runtime:

LOAD MYSQL VARIABLES TO RUNTIME;

SAVE MYSQL VARIABLES TO DISK;#save to disk

Now, lets confirm proxysql can access the mysql server by looking at the monitoring log:

SELECT * FROM monitor.mysql_server_connect_log ORDER BY time_start_us DESC LIMIT 10;

should show something like

+----------------+------+------------------+-------------------------+---------------------------------------------------------------------+

+----------------+------+------------------+-------------------------+---------------------------------------------------------------------+

| 192.168.99.100 | 3306 | 1526302545856645 | 1388 | NULL |

| 192.168.99.100 | 3306 | 1526302485856208 | 622 | NULL |

which is good, and we are happy with the mysql backend configuration so we load this config into runtime and save to disk.

LOAD MYSQL SERVERS TO RUNTIME;

SAVE MYSQL SERVERS TO DISK;

Lets create mysql users now:

INSERT INTO mysql_users(username,password,default_hostgroup) VALUES ('muser','muser',1);

and load users to runtime and update disk:

LOAD MYSQL USERS TO RUNTIME;LOAD MYSQL SERVERS TO RUNTIME;

SAVE MYSQL USERS FROM RUNTIME;

Now lets try a mysql dump (these instructions are not my own, these are excerpts from the proxysql blog, i have just reproduced this here for the sake of showing this working with my setup)

mysqldump --user=muser --password=muser -h 192.168.99.100 -P6033 employees salaries --skip-extended-insert | less

notice the output shows insert statements as

INSERT INTO `salaries` VALUES (10001,60117,'1986-06-26','1987-06-26');

INSERT INTO `salaries` VALUES (10001,62102,'1987-06-26','1988-06-25');

INSERT INTO `salaries` VALUES (10001,66074,'1988-06-25','1989-06-25');

Now lets create a rule to mask these salaries.

We will need to connect to our admin interface and add the rules:

INSERT INTO mysql_query_rules (rule_id,active,schemaname,match_pattern,replace_pattern) VALUES (1,1,'employees','^SELECT /\*!40001 SQL_NO_CACHE \*/ \* FROM `salaries`','SELECT SQL_NO_CACHE emp_no, 99999, from_date, to_date FROM salaries');

Load the rules to runtime

LOAD MYSQL QUERY RULES TO RUNTIME;

and save to disk:

SAVE MYSQL QUERY RULES TO DISK;

Exit the admin interaface and run the mysqldump again.

mysqldump --user=muser --password=muser -h 192.168.99.100 -P6033 employees salaries --skip-extended-insert | less

notice the salary is now masked and output now contains:

INSERT INTO `salaries` VALUES (10001,99999,'1986-06-26','1987-06-26');

INSERT INTO `salaries` VALUES (10001,99999,'1987-06-26','1988-06-25');

INSERT INTO `salaries` VALUES (10001,99999,'1988-06-25','1989-06-25');

Now we can improve on these and add more rules, but do not forget to load rules to runtime and to save for future save them to disk. There is also a handy script that allows you to create obfuscation rules for the columns you specify, this can be found at: https://gist.github.com/lefred/c040fee7e9c60ff3ca80f1590c48572b

How to find differences between two docker images or containers.

2017-08-17T16:45:00.001+01:00

Docker comes with a handy command called docker diff which allows you to find differences between the image and the container.

So if we make changes inside the container then we can find the difference from the image this is based on using the docker diff command.

But sometimes we might have to find differences between two running containers, in order to do this i found this helpful.

We can use the docker export command to export the filesystem of the container into a tarball and use our normal diff tools to compare.

This is how i did this:

Assume the image we are comparing is in our own docker registry docker.mysite.com:dev:0.25 and docker.mysite.com:dev:0.26 I would first create a container with the two images and then give the command

docker export dev025-instance -o dev05.tar

and

docker export dev026-instance -o dev06.tar

untar the files in separate folders and use an ide like PHPStorm to compare side by side.

For mac users there is a brew available to tap (https://github.com/moul/docker-diff) using

brew tap moul/moul

and then

brew install docker-diff

This does speed things up and you do not need to create containers etc, because docker-diff does that for you implicitly. However i felt it was easier with docker export commands due to large number of changes i was dealing with.

Docker diff works both with custom as well as official docker repos.

Setting up a drupal development env using docker image on mac os.

2016-03-07T22:27:00.002+00:00

I have been trying to try various ways in which i can make building and destroying development environments quickly and also make them as close to production as possible, i have tried using vagrant before till i was bitten by the docker bug.

There is an official docker image for drupal 8 and drupal 8 on the docker hub, i have tried using them before it wasn't a great wow moment to be honest.

Creating a drupal image using official drupal docker image form docker hub.

The docker image did not have drush or drupal console so it was a bit disappointing. I searched and came across https://hub.docker.com/r/iiiepe/nginx-drupal/ The guys have done a lot of work in using docker in their deployment workflow, so i wanted to give this a try.

I could have used docker hub to pull the iiiepe image, but i wanted to see what is inside and if i need to make changes i could.

I first created a folder in my docker subfolder, I normally keep docker images and my projects separate. I checked out the git repo https://github.com/iiiepe/docker-nginx-drupal

Obviously you need to have docker installed and configured properly, you can find a lot of tutorials for this on the internet, but basically you need to start your default docker machine i.e.

docker-machine start default

then, set the env like so

docker-machine env default

and type the command with eval() in the output of docker-machine env default command.

Now i am ready to create the docker images as below:

I went to the folder which has the dockerfile, then gave the following command

docker build -t drupalprogrammer/nginx-drupal .

This will pull appropriate images and begin to build images, in a few mins this will be all done and you can check the images by giving the docker images command

We will also need a mysql image for our drupal container to work with, we could have gone for the official mysql image however it doesnt work very well with os x because we want to keep the data outside the container and there is some work required to create scripts to allow this to happen.

There is instructions on the official mysql images page i.e. https://hub.docker.com/_/mysql/ i tried to use the instructions to load the data outside but could not get it to work. Luckily little googling led me to another image we can use.

run the following command:

docker run --name mysql-data -v /Users/nidash/data:/var/lib/mysql -p 3306:3306 -d dgraziotin/mysql

This will pull dgraziotin/mysql image and expose the 3306 port to use.

Now give the following command docker ps this shows the some-mysql container working, great job.

The first time you create the container this will also create an additional user called admin with a random generated password, to find the password you need to look at the docker logs like so.

docker logs mysql-data

So this shows the password like below:

You can now connect to this MySQL Server using: mysql -uadmin -p47nnf4FweaKu -h -P Please remember to change the above password as soon as possible! MySQL user 'root' has no password but only allows local connections

Using a mysql client connect to the mysql server and change your password eg.

SET PASSWORD FOR 'admin'@'%' = PASSWORD('password');

Now we need to the drupal container to be up as well, so here is the command to use.

docker run -d -p 8000:80 -v /Users/nidash/work/drupal7/:/var/www -v /Users/nidash/work/drupal7/sites/all/default/files:/var/www/sites/default/files drupalprogrammer/nginx-drupal

We need to map 2 volumes, /Users/nidash/work/drupal7 or where the codebase is to /var/www and because drupal is drupal and has files inside the code base, we need to map and mount this separately hence the other -v.

Complete the installation and that is it. Ensure the database host name is set as the IP the docker-machine env default command reports.

Creating a drupal 8 container from scratch.

2016-03-01T19:29:00.003+00:00

I tried the official drupal container according to https://www.drupal.org/node/2538074

Was'nt happy with the result so i thought build one from scratch.

What i want my drupal container to do?

1. Use my codebase which will be outside the container
2. Able to install drupal - correct file system permissions.
3. Clean urls should work
4. Drush & drupal console installed
5. Xdebug installed
6. Xhprof or any other profiler installed (optional)

Anything else?

Create a drupal development environment using docker.

2016-02-23T22:35:00.000+00:00

Currently at work i am using docker to streamline and speedup of development and deployment to testing and production, I am using symfony currently and our app is dockerised, i wanted to do the same for drupal based apps.

I searched for a official drupal docker repo and i did find this. https://www.drupal.org/node/2538074

Unfortunately there was very terse explanation but pretty much worked, i found a few hiccups along the way so wanted to share what i did.

Pre-requisite:
Install docker for your environment. Lots of details are available at https://docs.docker.com/ explaing docker and installing docker for your environment.

The docker repo holds the drupal container at https://hub.docker.com/_/drupal/, following the instructions this is how to create the drupal container:

In order to use my code base we need to mount the volume with the -v directive as below, we do this in one command:

docker run -v /$HOME/work/drupal8:/var/www/html -p 8080:80 --name mydrupal -d drupal

this will create a drupal container using drupal code in your home folder, this brought up a new site however soon i had a problem where

Because i was using my local folders mapped to /var/www/html i need to ensure the sites/default/files are writable, normally you should be careful what permissions you should set, more information is on drupal site. for now 777 will do, so i did a chmod 777 $HOME/work/drupal8/sites/default and also created a settings.php as instructed. Also chmod 777 $HOME/work/drupal8/sites/default/files.

Now comes the database part, at this point i created another container with mysql, fortunately a official mysql repo is also available on hub.docker.com

docker run --name mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -p 3307:3306 -d mysql:5.7

giving the command above download the mysql 5.7 image, set the root password, expose the mysql port in the container externally, now our drupal app can use this container. normally you would store the data outside the mysql container, the repo page of mysql on the docker hub discusses that, for demonstration purpose it is just fine to have out data inside.

i also connected to this database container using sequelPro and created a new database to use and named it drupal8, then continued with the installation.

Unfortunately then drupal died, with the following error:

I can again gave permissions, but ended again with another permission issue. Thank God i am using docker...so i dropped the drupal container and decided to mount modules and themes folder seperately, and use drupal core inside the container.

So first i drop the container as below:

docker rm -f mydrupal

so my modified docker run now becomes:

docker run -v /$HOME/work/drupal8/modules:/var/www/html/modules -v /$HOME/work/drupal8/themes:/var/www/html/themes -p 8080:80 --name mydrupal -d drupal

Notice the use of -v to use external folders for modules & themes otherwise use the drupal codebase inside the drupal container.

now reinstalling again. I had to drop the database and recreate a blank database and finish the install.

Somethings to remember, us the docker machine IP address, which you get by running docker-machine env default, use this IP address in the database config as well as the browser to look at your new drupal site!

All Done!

Use Symfony2 to create template/skeleton code

2015-09-13T20:51:00.000+01:00

Pre-requisite: A lamp environment, i will be using a vagrant lamp box (scotch-box) i created earlier.

To understand Symfony2 there is nothing better than getting ones hands dirty. We will try to use doctrine to build models, create views using twig and maybe use a dash of frontend magic using backbone or something similar.

For the test project I chose a sales bundle where we will make models for item, item type and sale.

Using symfony2 command line to create a model.

Because i have used vagrant scotch-box to provision my dev environment, i need to first make sure i do vagrant up to start the VM. then i do vagrant ssh this will take me to the ssh

locate where you have symfony installed, in my case it is /var/www/public/sandbox and give the following command:

php app/console generate:bundle

pick a name any name....well not actually, we need to be following coding standards set out in the symfony guide to ensure we name our bundle like this Bundle, so in my case i didn't have a vendor, so i chose a project name in this case it was Test\SandboxBundle, the bundle was named TestSandboxBundle and relevant files were created for me.

I just chose the defaults and had symfony create all it deemed good and wholesome...this will be additionally helpful if we peep under the hood as to what and where things are normally stored.

Now our bundle is created, we can create some models so in the symfony root in my case /var/www/public/sandbox i type the following command in order to generate the entities.

Entities we need: We are attempting to create a pos system, so we will have the following models:

ItemType - Types of item eg. Toys, Bakery, Drinks etc
Id, Name (string:50) & Description (string: 250)

Item - The item, this will linked with item type
Id, ItemTypeId (int), Name (string:50), Description (string:250), Price (descimal), Opening Stock (int)

Sale - the actual sale which will link to the item.
Id, Date (date time), ItemId (int), Quantity(int), CustomerId(int)

Customer - the customer, to make things easier, we will create only 2 customers 1 pos & the other for WEB orders.
Id(int), Name (string:50), Address (string:250), postcode (string:12)

So we generate these models one by one, the options i have chosen are as follows:

The Entity shortcut name: TestSandboxBundle:

Configuration format for all (yml, xml, php, or annotation) [annotation]: annotation

Note that the primary key will be added automatically (named id). When it asks do you want to generate an empty repository class say yes.

You can now start using the generated code!

Installing sqlyog on Mac OS X Yosemite

2015-09-13T16:45:00.001+01:00

I like to use sqlyog for my work with mysql, but only windows binaries are supplied on the google code site.

Another option is to use sqlyog in wine, so to install wine a simpler option is to use Homebrew (http://brew.sh/).

Installation for homebrew is quite straightforward, however you need to have installed the command line tools with xcode, if not please install that first.

The installation instructions are already on the first page of the homebrew website, so i will not reproduce this, once brew is installed, install wine.

brew install wine

when i tried installing wine, brew asked me to install a dependency and nicely told me to do the following

wine: XQuartz is required to install this formula.

You can install with Homebrew Cask:

brew install Caskroom/cask/xquartz

so i just did that, once this was completed i could complete installing wine.

Now head over to https://code.google.com/p/sqlyog/wiki/Downloads and install the latest sqlyog, at the time of writing this blog the version was 12.0.9 (SQLyog Community Edition - 12.09 (32-Bit))

Download the installer file, this will be in your Downloads folder. Use terminal and navigate to Downloads folder and install using the following command.

wine SQLyog-12.0.9-0.x86Community.exe

This installed sqlyog on my mac.

Installing Symfony2

2015-09-06T22:12:00.000+01:00

Installation for Symfony was straight forward, using the new symfony installer.

head over to symfony https://symfony.com/download and download the installer, installation instructions are included on the page.

Remember we are using the vagrant box we installed earlier, start vagrant using vagrant up and you should then have a public folder under your scotch-box folder under vagrant.

Install symfony under this folder.

Using Vagrant and Puppet to build a PHP/Lamp environment

2015-08-30T22:57:00.001+01:00

Vagrant is a real help in creating developer environments and is widely used by many development teams due to its flexibility and ease of replicating environments.

In order to get ready with Drupal 8, I wanted to get to know Symfony this is an attempt to do that and hopefully this will help me do just that and help others who stumble upon this site.

The steps for creating a development environment is below, I am using a Mac OS X Yosemite, you can use your favourite OS :

1. Install vagrant using vagrantup.com

2. Install a provider I have used VirtualBox.

3. Initialise a Vagrant environment.

Vagrant should work form any where on your computer because the installer would have added this to your executable paths, so lets create a new folder called 'vagrant' and run the following command

vagrant init hashicorp/precise64

This creates the Vagrantfile which is the main configuration file for vagrant. Open the Vagrantfile in your favourite editor or maybe just vi it. some of the configurations are worth noting, you can set here the port you want to forward, shared folders etc.

by default the line config.vm.box = "hashicorp/precise64" is uncommented, and this is the only line which is required to start your vagrant machine!

We will uncomment one more line

config.vm.network "forwarded_port", guest: 80, host: 8080

This will allow us to use our computer to access a website on the VM.

in the folder where the Vagrantfile is created do "vagrant up" thats it the vagrant machine is up albeit a very basic one...we will learn to it do all the things we want for symfony to work next.

This is how we can get Vagrant to setup a development box very quickly, now think this is a vanilla base installation, now you can configure this box, install a provisioner like puppet and create a base box for your specific development team needs.

Fortunately there are a lot of base boxes already setup with the required software for symfony to work, we can use them to start...one such base box i found at:

https://github.com/scotch-io/scotch-box

with instructions on how to setup this vagrant box, installation was a doddle and i was able to setup a vagrant box in under 40 mins.

Install the right version of php_mongo.dll in windows

2013-01-02T22:07:00.000+00:00

I use a macbook pro for my freelance work, a ubuntu netbook while travelling and forced to use a windows box at work.

I have been recently working a lot with Mongo DB and Drupal, I installed Mongo and php drivers for mongodb without any problems on Mac and Linux, I struggled to get the correct version of php_mongo.dll on my windows box. Installing php_mongo.dll becomes very simple if you find the right version of the dll file.

To find the right version, you need to ensure you look at your current configuration, note these steps these will be helpful.

use phpinfo() to find out the following information:

PHP Version

Compiler

Architecture

Server Software

For instance my local machine was a x64 machine running windows 7 pro, however the version of PHP installed was a 32bit. By phpinfo() shows me following:

PHP Version: 5.4.x

Compiler: MSVC9 (Visual C++ 2008)

Architecture: x86

SERVER_SOFTWARE: Apache/2.4.3 (Win32) OpenSSL/1.0.1c PHP/5.4.7

Notice from above I am running PHP major version "5.4", compiled with "VC9", with "x86" architecture and "Win32" (server software).

If you download the php mongo windows pre compiled binary from https://github.com/mongodb/mongo-php-driver/downloads extract and look at various files find thread safe version with 5.4 and VC9.

Hope this helps someone.

Import data from mysql to mongo

2012-12-21T18:28:00.003+00:00

To import data from mysql into mongodb we need to export data from mysql in a format mongo likes, this is either a tsv or a csv. Ideally you would like to use the following sql

select columns INTO OUTFILE '/path/to/csv'

FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'

LINES TERMINATED BY '\n' from table [where clause]

What the above SQL does is exports the data (only data , no headers) into a file as provided in your path, make sure you have access to location where you are putting the file. If you ignore the path and just use the filename this will be stored in the location where mysql data files are stored.

To get the fields in the data file, we can use the following

echo "SELECT * FROM
;" | mysql -uXXX -pXXX > '/path/to/

.tsv'

Finally you want all the be imported into a mongo database, here since our file contains a header we specify the --headerline option. For a complete list of options available with mongoimport please look at the mongodb site (http://docs.mongodb.org/manual/reference/mongoimport/)

mongoimport -d -c -type tsv --headerline

The headerline option uses the first line for fieldnames, this is fine till you have no data in your mysql and still want to import, then you are back to using the first option on OUTFILE but you also need to have a header file generated which you can then use in mongo, lets see how we can do this.

If you run into problems because of an empty data set, an empty set means no column names! use the steps below:

echo "SELECT column_name FROM information_Schema.cloumns WHERE table_schema = DATABASE() AND table_name='';" | mysql -uXXX -pXXX > '/path/to/_headers.tsv'

Then when importing to mongo use the following:

mongoimport -d -c -type tsv --fieldFile /path/to/_header.tsv

Use feeds module to import XML data in Drupal 7

2012-12-09T18:22:00.001+00:00

Recently i had to setup a xml import process to import data from an external supplier on a regular basis, the filename and format was not changing and i could choose where to store the file. Following is how i set it up.

First we need to download the following modules

feeds
feeds admin ui
feeds_xpathparser
ctools
job_scheduler

enable the above modules.

We will need a sample XML so this will be the one we will use for this demonstration:

<?xml version="1.0" encoding="utf-8" ?>

<Orders>
<Order>
<CustomerID>10</CustomerID>
<Order_Number>1</Order_Number>
<Order_Total>100</Order_Total>
<Currency>£</Currency>
</Order>
<Order>
<CustomerID>11</CustomerID>
<Order_Number>2</Order_Number>
<Order_Total>40</Order_Total>
<Currency>$</Currency>
</Order>
</Orders>

Now create a content type to hold this information. I am creating a content type with name as "order_info" give some description, to make things easier i have done following settings, removed body field, removed comments, removed 'publish to frontpage' options.

Now add fields as per above sample xml files we require the following fields

CustomerID, Text, 10
Order_Number, Text, 10
Order_Total, Decimal, 10.2
Currency, Text 3

Now that this is done now create an importer using structure->feeds_imports->add importer. We will name the importer "Order info xml importer".

Base the importer on following settings:

Basic settings
===========
Attach to content type: Order_info
Periodic import: As often as possible
Import on submission: uncheck
Click "Save" important to do this at each step so you do not loose the settings.

Fetcher
======
File upload - select
Click "Save"
got to settings and select "Supply path to file or directory directly" as we want to setup a regular import from a selected location. We also assume filename will be the same and will be located in the "site/default/files/feeds/" and we will call this sample.xml

Parser
=====
select XPath XML Parser
Click "Save"

Parser->Settings
=====   =======
This setting is for specifying your path information, define what is the context and fields, context is the row of data for our example context is "Order".

Context: Order

Following will be based on your content type, this is the place where you map your fields to data in XML.

field_order_number: Order_Number
field_customerid: CustomerID
field_order_total: Order_Total
field_currency: Currency
title: Order_Number

Here we have used title as identifying and unique information, and we are using Order_Number

Processor->Node processor->Settings
========   ============   ======
Content type: Order_info
Click "Save"

Processor->Node processor->Mapping
========   ============   ======
Use source as "Xpath Expression" and target as the XML data item, when you add this will add mapping as below:

xpathparser: =>

After you have completed the assignments the mappings will look as follows:

xpathparser:0 => Order_Number
xpathparser:1 => CustomerID
xpathparser:2 => Order_Total
xpathparser:3 => Currency

xpathparser:4 => Title => Unique

As we need to identify an item in XML as unique to ensure data integrity, we can choose title of node and make sure they are marked as unique. Later we will use order_number and put this in the Title, this will also help identify the content.

After saving this, now lets import. For this we need to go to "http://yourlocal/import" and select 'Order_info' put the title as 'Order XML Import' this will create a content that allows your to import data into your Order_info content type, here you need to select location of file lets put "public://feeds/Sample.xml" and put title as "Order XML Import" and save the content.

To Import manually click 'Import' or to see how this will be imported using cron go to admin/config/system/cron and see new nodes being created from you XML.

Would you love apt-get on a mac?

2012-12-04T21:58:00.001+00:00

As Ubuntu kept maturing i kept on gradually liking the "apt-get install" command in ubuntu to the point this felt like a hinderance on other linux flavours more notably Mac.

To my surprise there is a tool available to make apt-get work in mac, this is fink (http://www.finkproject.org). The binary is available to download for 10.5 and lower, for the rest who have moved on it seems the only option is to compile and install from source as per the instructions on the fink website.

I had 10.8 (Mountain Lion) installed and as the fink site suggests i had to compile using source, before i could install fink i had to install xcode and xcode developer tools (xcode > preferences > downloads > developer tools)

xcode-select -switch /path/to/Xcode.app/Contents/Developer

I had to accept a license for Xcode build by reading and typing 'accept'

sudo xcodebuild -license

Unpack tar.gz manually or double click in your browsers download, if you want to manually uncompress

cd $HOME/Downloads

followed by

tar -xvf fink-0.34.4.tar.gz

tar -xvf fink-0.34.4.tar

where the choice depends on whether the source has already been partially unpacked, e.g. by Safari.

Then run the following commands in a terminal window:

cd /Users//Downloads/fink-0.34.4

or wherever you have downloaded and un-packed fink and run the following command:

sudo ./bootstrap /Applications/fink

Sit back, relax and keep glancing at your terminal window to choose options at prompts, for most options default is fine, for geographic locations you may want to select the one closest to you.

After the installation is completed, running the command without sudo - this will comeup with a prompt this will allow you to configure apt-get to work from any part of your system.

/Applications/fink/bin/pathsetup.sh

After this do the following:

fink selfupdate-rsync

fink index -f

Once this is completed you should be able to use apt-get in mac!!!

Sqlite for drupal 7

2012-04-04T23:10:00.001+01:00

Recently one of my acquaintances started his own business. He was on a shoe string budget and did not want to pay for a mysql database. I toyed with using free mysql hosting offering and tried one on amazon cloud (xeround).
I was able to make it work but the performance was very bad. With luck i found using phpinfo() that sqllite support was built in and installed the site in under 10mins. Cool ha!

Defenitive guide to drupal 7 at a discount...

2012-02-05T17:25:00.000+00:00

One of the must have books i have every read on Drupal 7, the collection of authors have done a marvellous job. The usual price from amazon.co.uk is around £28 but using the above link you can get it for £19 a saving of around £10.

There are many a book reviews about this book so i am not going to get into this, i have just started reading the book. One thing the book with the link i have provided is a warehouse clearance deal, which means the book is not hot of the press. but i bought from the warehouse deals myself, and i could not see any difference. Advised to check details on the items before you buy.

Google Web Fonts with Drupal

2012-02-04T11:08:00.000+00:00

Google webfonts is a really nice and easy way to spice up your website, gone are the days of fiddling with never failsafe attempts with the likes of cufon, this is really nice and easy.

And a good thing like always there is a drupal module for it. Just download the @font-your-face module and configure, you can choose which fonts you can enable and choose which font to apply to what kind of html tags, you can choose a separate font for h1, h2, h3, h4, h5, h6 and separate fonts for all text, but not the best of interface, but does the job.

The hardest thing was after installation trying to find where to do the settings, quick look at the documentation and it was found that just go to appearance and it adds a tab for font-your-face.

I really like this and have tried this on my drupal commerce demo site..

http://www.grayns.co.uk/online/

I have used the Open Sans 700italic (cyrillic) for headers and Amaranth 400italic (latin) for everything else...was easy breezy.

working with git

2012-01-16T22:51:00.001+00:00

Build online shop using Drupal commerce

2012-01-01T14:03:00.000+00:00

I have built a commerce demo site by using Drupal Commerce and Drupal 7.9, it was a breeze to setup but bit difficult to customise.

I have used various types of products types and a single product display to manage all the content and products. The theme is currently 'bartik' which is the default theme that comes with Drupal 7.

I enjoyed setting this shop up, i have used views to customise product pages, used taxonomy terms in the department and product pages to improve SEO and display of pages.

http://www.grayns.co.uk/online/

Online Demo of an E-commerce site using Drupal commerce.

Does NoSQL means you forget everything you learnt at RDBMS school?

2011-10-14T21:54:00.000+01:00

I found it interesting to see how to integrate NoSQL, but more interesting was to unlearn somethings i learnt through years of training - denormalise! Normalisation was the term that made rdbms, good normalisation design was supposed to be a hallmark of a good database designer. But NoSQL is different, forget joins forget collecting data from 10 different tables NoSQL is radically different and the best part is it works for very large datasets, is distributed, and some very big installations like twitter, amazon and others mean if you don't look at this seriously soon you will be in trouble.

Drupal by itself doesn't support NoSQL out of the box and you don't throw away mysql yet, there is a module (cassandra) that was recently added on drupal.org which is worth checking, One possible way of integrating with drupal is using a nosql may be casandra or mongodb with your entities, infact this is one of the best things of entities, your fields can live in a separate database altogether and still integrate with drupal sub system.

I plan to put a mongodb/entities/drupal integration on the blog soon, as soon as i get some time away from some projects i am working on.

My Macbook informs me of tragic death of Steve Jobs the Apple co-founder

2011-10-06T19:18:00.000+01:00

Like everyone in the UK i woke up to the news of tragic death of Steve Jobs the Apple co-founder, i was not going to write a blog post on the topic, but there was something that prompted me to write one.

I got the news not from my tv but my macbook pro! I switched on my macbook, fired up a new tab in safari, and there it was, very simple, very sober 'Steve Jobs' 1955-2011. A real tribute to the design genius he was. I am sure there will be a new person at the helm of things at Apple, Apple may continue to grow in leaps and bounds or may just fizzle out - but you can't take that away from Steve, the way he came back to Apple when things weren't going very well - brought the iMac which was a run away success, followed up with other fantastic products we all love and cherish.

The most important thing was he got Apple to believe they were not in the business to shift boxes but deliver what the customer expects, stuff that just works.

Hats off and tributes to the genius…

Configuring drush on Mac OS X Lion

2011-09-30T21:59:00.000+01:00

I downloaded the drush from the drupal website at http://drupal.org/project/drush. I downloaded the 7.x-4.5 version - I was using mac os x and if you are also using mac os or any other flavour of linux - you should download the tar.gz file.

Drush works best on linux/mac os x - it works on windows as well but has its limitations, although things are improving but you will need to install additional software that allows you to use many drush functions. If you are interested to look at how to install drush on windows then have a look at drush.ws. I did install on my old windows pc as well, got most of it working.

Coming back to the topic, we are installing on mac os x - so here goes.

The accompanying readme.txt file is quite good in showing what is required, and it really is straight forward. Albeit there are a few differences and may be you would like to customise your installation a little bit - but this is what i did.

The first step was to download and extract the drush-7.x-4.5.tar.gz file to a folder out of web root as recommended.

Then make 'drush' executable, I installed drush in my home folder hence my command to make drush executable was

chmod u+x ~/drush/drush

a quick jump to ~/drush folder and enter drush at the $ prompt, as you have made drush executable your should see drush options, a good way to learn options is to use drush help.

if all is fine, we can create a symlink to /usr/bin so that drush is always in the path. The command to do this of course is

ln -s ~/drush/drush /usr/bin/drush

you may require sudo access so prepend sudo to the above command.

Thats all, i tried drush and all worked swell! I was able to download drupal, install modules, enable/disable modules and everything. I hit a block when i was trying site-install option.Drush has a powerful option that lets you install a new site from the command prompt itself, and it is extremely fast.

The usual syntax for site-install is as follows

drush si standard --db-url=mysql://root:xxx@localhost/dru7 --site-name="Drupal Example" --site-mail=xyz@xyz.com

I got an error which was

sh: mysql command not found.

Thankfully it was an easy fix, it was because though mysql was installed and working, mysql was not working from the path where drush was installed. so symlinking to the rescue i did the following

ln -s /usr/local/mysql/bin/mysql /usr/bin/mysql

Thats it - all working fine now and i am enjoying working with drush.