JarodMS Drupal and Dev

Windows, Nginx, MariaDB...and sometimes SQLServer

2015-06-22T14:37:00.000-05:00

I have a couple projects at work that are quite different.

All are using PHP.

Some projects require connecting to MSSql (SQL Server).

Other projects are Drupal based.

I had been using XAMPP for awhile now, but knew I wanted to use Nginx on Windows, so I finally took the leap and got it running.

Here's a great stack to use for Windows, PHP, Nginx, XDebug, and MariaDB: http://wpn-xm.org/

Now, in my case I needed to interface with a SQL Server database for other projects. The Microsoft Drivers for PHP for SQL Server are located here: https://msdn.microsoft.com/en-us/sqlserver/ff657782.aspx

BUT (at the time of this writing) PHP 32-bit is required for these SQL Server drivers, so make sure you download the correct package from WPN-XM. This was my "gotcha" that took awhile for me to get this running. I was getting "could not find driver" even though the correct versions were available.

I've got some more learning to do, but so far I've got multiple projects running with some accessing MSSQL and others MariaDB, all using Nginx and PHP. The speed difference compared to Apache is remarkably noticeable.

Also for my Drupal projects, I was able to do a simple export from MySQL and then an import into MariaDB. My Drupal sites needed no changes in-order-to run, other than a server config in the nginx.conf file.

Python example to Add task in Todoist

2015-05-20T14:31:00.004-05:00

A simple script that will prompt you for a task description and date, then add it to the project in Todoist.

Requires:
* https://github.com/Doist/todoist-python - View the Readme for installing the todoist-python library
* Python 2.7+
* PIP: https://pypi.python.org/pypi/pip

Code:

import todoist

api = todoist.TodoistAPI('HERE_IS_MY_API_KEY')
task_desc = input('Enter the task: ')
task_time = input('When (optional): ')
item = api.items.add(task_desc, MY_PROJECT_ID, date_string=task_time)
api.commit()

I just created a simple shortcut in my taskbar and it easily adds tasks to the project ID. I used the Inbox project Id from my account so everything goes in there. I can sort it later.

The "When" parameter recognizes the Todoist date/times such as:

today
tomorrow
2 weeks
1 month

If you press Enter, instead of adding a date, then the task is still added, but with no date.

Multifield - Add value using web service

2015-04-21T10:45:00.000-05:00

One little checkbox...that's all it took. Something I didn't understand, but now do just a little better.

Here we are:

Using Multifield module

Multifield is defined as Unlimited Number of values

Using Services with REST server to update a node - adding a value in the list of Multifields
Using Postman to PUT to update that node
Content type name: test_multi
Multifield field name: field_jarod_multi
Multifield subfield name: field_jarod_note, Integer

Here's what the Multifield config looks like by default for jarod_multi:

Note the checkbox is on for "Hide blank items". So when editing a node we see this:

That's fine. If I want to add another, I just use the button and we're good.

BUT it doesn't work when adding another through web services. Doing a PUT will give me a return code of 200, but the value is never added.

To do that, uncheck the "Hide blank items" button which gives a view like this when editing a node:

Now in my web services I do a PUT:

The full value for the Key is

node[field_jarod_multi][und][2][field_jarod_note][und][0][value]

After the PUT, I get the new multifield value added to the node:

Lots of headaches over this one. I just missed the checkbox. So glad this one is solved.

SMTP Auth Support - Drupal module Gotcha!

2015-02-04T11:43:00.003-06:00

In using the SMTP module, I found an error that I really did not expect.

Using the Debug mode of SMTP Auth, I found that the email/password combination I was using was working ok. Auth was always successful. When sending the message, I continued to get "553 Relaying disallowed".

The options for "E-MAIL OPTIONS -> E-mail from address" in the SMTP auth config page were all correct. It should be using this from address to send emails. Nope!

I don't know what made me check the next thing, but I went to "System -> Site Information" and changed the From email address to the one I was using in SMTP auth. Abracadabra, email sending worked.

My Drupal Web Stack - new and improved

2014-10-20T00:11:00.003-05:00

It's amazing how sometimes breaking through a major obstacle leads to new and improved technologies.

I had recently realized that one of my servers had been compromised. Hacked into. The best solution I had come up with was to rebuild the server. I quickly did a backup of the databases and all the code and moved them off of the server.

When spinning up a new server, I had the chance to finally move to a different tech stack I had been wanting to use.

CentOS 7
Nginx
MariaDB

This meant no more Apache and no more MySQL. The change between MySQL and MariaDB was really only in the name. There was no change in code or scripts for what I was doing. Awesome!

The Nginx implementation was a lot easier than I thought it was going to be too. I had some Nginx experience from a past contract and knew of the performance increase. Some configuration changes and learning how to setup multiple virtual hosts, and that was that.

At this point, the basic installation seems as fast as my previous Apache + Memcache install.

What is left?

Well this was a very quick implementation in-order-to get my websites moved to a different server. As fast as the stack is right now, I still want to install Memcache as well as Varnish.

Here are a couple of great articles that I used to setup my server and get Drupal 7 running with Nginx and MariaDB.

https://www.rosehosting.com/blog/how-to-install-lemp-linux-nginx-mariadb-php-fpm-on-a-centos-7-vps/

https://www.rosehosting.com/blog/how-to-install-drupal-7-on-centos-7-with-nginx-mariadb-and-php-fpm/

Don't forget to install Drush on your new server. It's too easy:
https://www.drupal.org/node/2132447

A little bit of learning, but I'm very happy with the LEMP and Drupal 7 configuration.

Backup and Migrate - Restore 1 Table

2014-08-07T14:24:00.002-05:00

Well I never thought about trying this before, and there was no reason why it wouldn't work, but restoring a single table into a Drupal website using the Backup and Migrate module worked like a charm!

Scenario

I'm using the IP Ranges module on several websites across several servers. Most of the time the websites have content that is specific to certain geographic regions or countries, so getting hit a ton by China, Russia, or Brazil isn't desirable.

Also, adding the IP Ranges entries to each website (or even a new one) is tedious.

Enter Backup and Migrate

On one site, I do all of my manual entries using the IP Ranges interface. Then I can do a backup of the IP Ranges table from this site. The backup will contain info for just IP_RANGES and nothing else. I do not want to break my other sites. I have a Settings Profile for IP Ranges and then I do a manual backup.

On another site, do a restore using that backup file. But wait.....read below.

One Gotcha!

The backup contains a "DROP TABLE IF EXISTS" statement for ip_ranges. Now the script should be fairly quick, but dropping the table while the module is enabled can cause issues for the site if someone visits.

What I like to do is replace the statement in the script with "TRUNCATE" giving: TRUNCATE `ip_ranges`;

double check the rest of the script to make sure you haven't accidentally included other stuff.

Then run the script through the rest of the sites!

I'd rather use Firewall rules, but I need to do more research on that. I've broken access to sites before because I've entered the wrong Firewall rule. :P

Blocking the Spammers

2014-07-23T00:40:00.004-05:00

Stop the spammers and bad referrers.

This is a compilation of info I found on various sites.

Add this to the end of your .htaccess file:

SetEnvIfNoCase Via evil-spam-proxy spammer=yes
SetEnvIfNoCase Referer evil-spam-domain.com spammer=yes
SetEnvIfNoCase Referer evil-spam-keyword spammer=yes
SetEnvIfNoCase Via pinappleproxy spammer=yes
SetEnvIfNoCase Referer semalt.com spammer=yes
SetEnvIfNoCase Referer semalt.semalt.com spammer=yes
SetEnvIfNoCase Referer poker spammer=yes

Order allow,deny
Allow from all
Deny from env=spammer

Drupal Modules I use:

ip_ranges - https://www.drupal.org/project/ip_ranges - Block entire range of IP's.

Additional Sites:

http://myip.ms/info/whois/ OR any other popular Whois database. I use this to see the range of IP's I want to exclude.

This process is a little long for me. Going through the logs and seeing what IP's are hitting bad URL's or protected URL's. Then find the IP range and enter into IP Ranges. I'd like to see a nice report that uses a Whois database to show where the requests are coming from.

I'm still looking for a better overall solution. For example, I've got to be able to put the rules somewhere in my Apache config to cover all the sites on my server, rather than configuring each one individually, I just haven't taken the time to do it yet. We'll see. :)

Module - Front Page

2014-06-11T12:52:00.001-05:00

Do you want different user roles to have different Front pages when they login or go to the Home page?

The Front Page module (https://drupal.org/project/front) works very well for that. 18,109 reported installs at the time of this writing.

If you want token support, 7.x-2.4 and lower will need a patch: https://drupal.org/node/1786128

Front Page handles roles in the reverse order of what you have defined in the Role arrangement and creation page (admin/people/permissions/roles), so Admin role, then Authenticated, and then finally Anonymous role will be processed when implementing a custom front page.

Here's a screenshot of 2 roles, Authenticated and Anonymous.

As an example, for an e-commerce site and with token support, I can redirect a "customer" role to their Orders page.

More advanced users and cases will probably want to use Rules.

Handle potential plural items

2013-04-22T14:50:00.001-05:00

Let's say you have a count of items ("widgets") that you want to display.

When you count them, the result could be 0, 1, or more than 1 item. SO you need to display one of the following:

0 widgets
1 widget
2 widgets

In PHP, you can handle this with the following code:

<?php print $view->total_rows;
if ($view->total_rows == 1) {
print " widget";
} else {
print " widgets";
}
?>

Or use Drupals built in function "format_plural" to help out:
<?php print format_plural(max(0, $view->total_rows), '1 widget', '@count widgets'); ?>

JSON for Drupal select list and location

2013-03-13T08:18:00.001-05:00

Background

I'm using PhoneGap and DrupalGap to create my first Android app called "2 For Deals" which helps you track and locate deals in stores that are "2 For" some dollar value. Location of the item, as well as expiration of the deal are tracked. Have a gander (with screenshots): https://play.google.com/store/apps/details?id=com.twofordeals

Tech Talk

DrupalGap provides a JavaScript library for your PhoneGap app, as well as a Drupal module which provides services for accessing data through the Drupal site/data. Great stuff if you want to combine the two.

JSON

Here is the JSON I used to create a new node which includes the following fields:

field_cost: A Decimal field type using the text field widget
field_select: A "List (text)" field type using the "Select list" widget. Notice the difference between this and some of the other fields. No index is used.
A simple Location determined by users' GPS. This is the location for the node, where 1 and only 1 is allowed. Not sure if this is correct, but I had to provide at a minimum the locations and location fields.
Body: standard body field used in Drupal

{
"node": {
"type": "my-content-type",
"title": "This is my new title ",
"language": "und",
"field_cost": {
"und": {
"0": {
"value": "5"
}
}
},
"field_select": {
"und": {
"value": "key-from-drupal for the select"
}
},
"locations": [{
"locpick": {
"user_latitude": "41.98464",
"user_longitude": "-91.673859"
}
}],
"location": {
"locpick": {
"user_latitude": "41.98464",
"user_longitude": "-91.673859"
}
},
"body": {
"und": {
"0": {
"value": "This is the body of my node"
}
}
}
}
}

For blogging purposes, some of the fields are made more generic. The location is one that exists, but points to the middle of a river. You won't find me there. :)

I'm posting this because different pieces caused me different headaches at times. For example on the mobile app, I wanted to collect as little data as possible about the location, so I needed to find the sweet spot of how to create the location on the server side with as few pieces of data as possible.

TODO: I will eventually be changing the Location to a node reference. Which means I will need to find the location(s) closest to the users location, display a list, then submit using that reference. Fun!!!

I hope this helps someone. Enjoy!

New Server setup with CentOS 6, Drush and Drupal libs

2012-12-19T12:57:00.001-06:00

My compilation of docs and tutorials to help me get a server setup with CentOS 6, Drush and various libs needed for Drupal 7. Once that is complete and working, a short write-up on getting a Drupal 7 site up and running in less than 2 minutes.

Webmin install and setup: http://jarodms-drupal.blogspot.com/2012/07/webmin-on-ec2.html
Config Firewall for Webmin: http://www.webmin.com/firewall.html
LAMP Setup: http://library.linode.com/lamp-guides/centos-6
Drush in less than a minute: http://danreb.com/content/how-install-drush-centos-linux-drupal-development
Additional libraries for Drupal:

yum install php-gd
yum install php-dom
yum install php-mbstring

Additional Steps:

Optimize php.ini
Depending on server, may need to open firewall for port 80

Install Drupal 7 instance in less than 2 minutes:

Create MySQL Database
Create MySQL User/pass
Grant MySQL privileges for user
Config virtual host for new Drupal site and restart Apache

<VirtualHost *:80>

DocumentRoot "/var/www/html/drupal7"
ServerName fill-in
ServerAlias fill-in
ErrorLog /var/www/logs/drupal7-error.log
CustomLog /var/www/logs/drupal7-access.log combined
<Directory "/var/www/html/drupal7">

RewriteEngine on

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !=/favicon.ico
RewriteRule ^(.*)$ index.php?q=$1 [L,QSA]

</Directory>

</VirtualHost>

Use Drush to download Drupal and install site:

drush dl
drush site-install standard --account-name=admin --account-pass=admin --db-url=mysql://SiteMySQLUser:SiteMySQLUserPassword@localhost/SiteMySQLDatabase

My Dell Inspiron 1764 - CentOS 6.3 and Broadcom Wireless BCM43224 driver

2012-12-07T09:15:00.002-06:00

There is no default driver for Broadcom wireless cards on CentOS 6.3. Broadcom is apparently getting better at Linux support, so they provide the source you can compile to get the driver. Here's the awesome and simple writeup that I found. It really did take just 5-10 minutes of copy/paste/execute.

I won't even try to recreate all the steps in this blog. Just go to http://elrepo.org/tiki/wl-kmod and follow the instructions. Worked like a charm!

My setup:
Dell Inspiron 1764 dual-boot into Windows 7 and CentOS 6.3
PCI Device (lspci at terminal) :
Network controller: Broadcom Corporation BCM43224 802.11a/b/g/n (rev 01)

Drupal User Admin module stack

2012-11-20T13:59:00.001-06:00

Here's a stack of modules that I like which help a non-administrator manage their own users.

Administer Users By Role: http://drupal.org/project/administerusersbyrole
Role Delegation: http://drupal.org/project/role_delegation
Login One Time: http://drupal.org/project/login_one_time
No Current Password: http://drupal.org/project/nocurrent_pass

Once installed and enabled, I create a role ("site-owner" for example) and assign it the various permissions.
A few more little tricks and now the Site-Owner can create or cancel users, assign various roles, and send them a One Time Login link if needed.

One caveat I need to investigate...the Administrators (e.g., User 1) are still shown on the list of users (admin/people) and certain roles can be added or removed, but the Administrator role can't be removed and the Admin account can't be edited or canceled. Really the Site Owner can't do anything harmful to the Admin, but I'd prefer that account wasn't shown at all in the list. I need a module that will only show the users whose role is less than or equal to the current role.

Drupal 7, Services 3.x , Request and Response

2012-10-18T17:38:00.000-05:00

Here is something that I just recently learned regarding request/response from a REST server.
In this case, I am using Drupal 7 with Services 3.x.

It's just one of those things I guess you don't lean until you need to, and I never had yet. It was difficult to find the info though.

Here is the scenario:

You are calling a web service using JSON as the Content Type.
You are getting back an XML response.
You want to get back a JSON response.

Here's what you need to do.

Goto your services

Select "Edit Server"

Make sure the response formatter "json" is selected. It may already be.

Now in your JSON REQUEST, add the following header:

Accept = "application/json"

For some reason, Drupal was returning the response in XML format by default, I don't know why, but adding this header to the request immediately gave me back a JSON response.

Webmin on EC2

2012-07-02T00:23:00.000-05:00

Installing the RPM package of Webmin on EC2 was a 5 step process for me:

*** Verify the latest RPM version at http://www.webmin.com/download.html

wget http://prdownloads.sourceforge.net/webadmin/webmin-1.610-1.noarch.rpm

sudo rpm -U webmin-1.610-1.noarch.rpm

sudo /usr/libexec/webmin/changepass.pl /etc/webmin root foo

Change the webmin root password to foo.

sudo /etc/init.d/webmin restart

Configure IPTables to allow port 10000: http://www.webmin.com/firewall.html

Webmin will be accessible at http://youserverip:10000

* Works fine on CentOS in Linode and Rackspace.

* Would be nice to install cert for this though.

What's New in Drupal 7

2012-04-16T15:29:00.001-05:00

What's New in Drupal 7 | drupal.org

A good page to read...just in case someone asks you. ;)

'via Blog this'

Getting Drupal updates into TEST

2012-03-30T08:44:00.000-05:00

I create a branch from TEST and call it something like updates. (git checkout -b updates)

Now I'm working in the branch updates

Create a backup of the database - important and discussed later.
Pull the necessary modules or Drupal updates and test it out.
Everything look ok? Then merge the branch to TEST.

git checkout test
git merge updates
handle any conflicts

Why would you do this rather than pull your Drupal updates right into TEST? Murphy's Law. If you pull the Drupal updates into TEST first, then somebody is bound to have an issue or item that needs worked on that needs to go into TEST before you had a chance to completely test and verify your Drupal upgrade.

The approach above helps you get around this. You're in the middle of testing Drupal updates and someone approaches you with a bug or change that needs to go into TEST first. How do you roll back to the TEST branch?

checkout TEST (git checkout test)
Restore the database using the backup you created earlier.
You are now at the TEST version you need for the new change, without those Drupal updates that you haven't completely tested.
Create your new bug/change branch and continue on the issue that was asked of you.

Drupal - Bootstrap for old PHP script

2012-02-27T10:20:00.002-06:00

In a hurry and I have an old PHP application that pulls data from proprietary tables. I have connected some of the data to a node, but now I need to display the data in a separate window when applicable.

So, using the old and large PHP script, just to get it functioning, I had to bootstrap Drupal.
Place this at the beginning of OldLarge.php

$drupal_dir = getenv("DOCUMENT_ROOT");
$current_directory = getcwd();
chdir($drupal_dir);

require_once './includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

...
Do my Drupal stuff here....db_query() and whatever so that I can get results back from weird tables.
...

Then at the end of OldLarge.php , add the final code....

chdir($current_directory);

Eventually...some day (which ends up being never) I will convert the OldLarge.php script to something more Drupalized.

CiviCRM "Javascript must be enabled in your browser in order to use the dashboard features"

2012-02-08T10:35:00.001-06:00

"Javascript must be enabled in your browser in order to use the dashboard features"

I started getting this error from some of my testers when they were using IE9. Sure enough, I couldn't access the CiviCRM dashboard or CiviCRM menu (it acted like it was disabled).

Yep, well JavaScript is enabled.

Stats on the site:
CiviCRM version 3.4.8
Latest Drupal (6.24)
Drupal module jquery_update installed (giving Drupal the jQuery version 1.3.2)

Hmmmmmm.....It works FINE with Chrome (16.0.912.77 m), Opera (11.61), Safari (5.1.2), and Firefox (10).

Oh YEAH! That's right!!! Internet Explorer doesn't handle a lot of JavaScript files very well. CSS files are limited to 32(?) I think. Not sure what the JS limit is.

So on the TEST site I enabled "Optimize JavaScript files" and all works well in IE8/IE9.

Things to look at:
http://drupal.org/project/javascript_aggregator
Great article by Vlad http://shvetsgroup.com/blog/optimizing-javascript-and-css-files-drupal

User Agents

2012-01-04T21:56:00.001-06:00

A large list of user agents
http://techpatterns.com/forums/about304.html

Any other lists you know of?

My current sites/all/modules

2011-12-16T14:51:00.000-06:00

My current "modules" folder and what it contains:

civicrm
contrib
custom
shortsitename_features

I kind of like that. Nice and sweet. There's a bunch of modules under "contrib" of course, a few under custom (but growing) and a few under shortsitename_features (but growing).

CiviCRM - moving your configuration changes to a different environment

2011-12-13T15:48:00.003-06:00

I love code driven development. So when I build my Drupal sites, I immediately download and enable Features. I do not want to handle website settings between environments by comparing screens and clicking box after box. When you are running in a local DEV environment, plus you have a TEST and LIVE environment, using Features saves time (which of course equals money).

So I started a new project using CiviCRM and the thought in the back of my mind for awhile was:

"I just changed a CiviCRM setting. How do I migrate that change to my TEST and LIVE environments?".

Well the time came and I had to do it. I needed to get my TEST site going and didn't want to bother remembering all of the settings I needed to configure.

Luckily, CiviCRM comes with migrate scripts to Export and Import your settings.

First Export it from the localhost (DEV environment) using one of the options. More exist, but these are pretty basic:

Browse to

http://localhost/virtual_dir/sites/all/modules/civicrm/bin/migrate/export.php?name=&pass=&key=

This will download a file called CustomGroupData.xml

Using wget (if you want to script this and update the file at 3pm for example)

wget -O D:\dev\htdocs\virtual_dir\sites\default\CiviCRM.info "http://localhost/virtual_dir/sites/all/modules/civicrm/bin/migrate/export.php?name=&pass=&key="

I save this file to a folder in the website I am working on. Then, if it has changed, I commit it to git.

After the export file has been committed to git, then moved to your next server, import:
http://example.com/sites/all/modules/civicrm/bin/migrate/import.php?name=&pass=&key=&file=../../../../../default/CiviCRM.info

Notice the difference between this command and previous ones is that it uses import.php and specifies a file.

I noticed there were a couple of settings that did not get exported and I think those are related to CiviCRM Profiles. I need to look into this a bit and see if this is an issue or "works as designed".

What are some other options for moving CiviCRM configuration changes between environments?

Drupal - Code Driven Development

2011-11-07T14:37:00.000-06:00

I think most of my sites have 2 versions...LOCAL and LIVE to allow testing of various changes as well as developing new themes and modules. Some sites have 3 versions, LOCAL, USER TEST, and LIVE.

I like having a local version so that I can polish it first. Also, I use tools like NetBeans for coding and debugging.

I use "Code driven development" for all my Drupal sites. Remember, the changes you are making through the GUI in your Drupal site are being saved to the database. To use code-driven development, you need to get those changes out of the DB and into code. This is where Features comes in. My process:

Develop code or theme; make your config changes in Drupal
Export as much as possible using Features and Strongarm - This puts the DB changes into code
Checkin the code to your version control system
On the server - pull code from your repo
Enable the various Features

Once you have your systems and procedures in place, this can actually be a time saver. It's actually quite fun!!!! :)

Debugging PHP in NetBeans (in Ubuntu)

2011-11-07T10:37:00.001-06:00

Here are the steps I used to setup PHP debugging in NetBeans (6.8 at the time of this writing) in Ubuntu 10.

sudo apt-get install php-pear
sudo apt-get install php5-dev
sudo apt-get install php5-xdebug

Edit your PHP INI settings
sudo nano /etc/php5/apache2/php.ini

Add the following lines under extensions:

zend_extension=/usr/lib/php5/the-folder/xdebug.so
xdebug.remote_enable=on

In my case it was ""/usr/lib/php5/20090626+lfs". You will have to find the xdebug.so file in the correct folder.

Restart Apache
sudo /etc/init.d/apache2 restart

NetBeans PHP Debug Settings

Goto tools->Options
Select PHP
On the General tab
If this is your first time configuring xdebug, you may want to keep "Stop at First Line" checked. this way when you start debugging you should know right away that your setup is correct.

Start Debugging

Add some break points.
Right-click your project and select Debug

CentOS 5.6 server setup

2011-10-13T13:16:00.000-05:00

These are the guides and blog posts I use to setup LAMP on my new CentOS servers:

Get a newer version of PHP

http://wiki.centos.org/HowTos/PHP_5.1_To_5.2

Setup RPM Key

cd /etc/pki/rpm-gpg
wget http://download.fedora.redhat.com/pub/epel/RPM-GPG-KEY-EPEL

Setup YUM repo for Git

http://www.davegardner.me.uk/blog/2010/01/29/setting-up-git-on-centos-5-server/#install

Install Apache, MySQL, PHP

http://library.linode.com/lamp-guides/centos-5

I typically install Webmin too, so I'll document that one in a bit.