Welcome To YakBiz

Troubleshooting NFS

2013-09-04T12:03:00.003-07:00

Common NFS Errors

"No such host" - Name of the server is specified incorrectly
"No such file or directory" - Either the local or remote file system is specified incorrectly.
"No such device" - NFS is not configured into the client's kernel.
"NFS server is not responding" message followed by "NFS server xxxx OK" - Server is heavily loaded causing RPC timeouts, or server has crashed.
"Stale file handle" - The file is no longer available.
"MOUNT_PROG not registered" - rpc.mountd daemon never started up and registered.
"Too many levels of remote in path" - Attempting to mount a file system which is already an NFS mounted file system.
"Permission denied" - Accessing as root on the client and root is mapped to nobody. Or the user on the client does not have corresponding UID on the server.
"No space" - The server is out of space on the file system.

Troubleshooting

The showmount command may be used to display server-side mount information. The option -a displays all remote mounts showing the name of the client and the directory, separated by a colon. The -d option displays only the names of the directories mounted by the clients. And the -e option displays the list of file systems exported by the server.

     # showmount -a
     All mount points on local host:
     edcert20.ucs.indiana.edu:/home
     edcert21.ucs.indiana.edu:/usr/local

     # showmount -d
     Directories on local host:
     /home
     /usr/local

     # showmount -e
     Export list on local host
     /home           edcert21.ucs.indiana.edu edcert20.ucs.indiana.edu     
     /usr/local      edcert21.ucs.indiana.edu

The df command may be used to display information on the file systems mounted remotely, the mount point and the amount of available space. The -F option may be specified to list only a specified file system type.

     # df -F nfs
     Filesystem                      Type  blocks     use   avail %use  Mounted on
     edcert21.ucs.indiana.edu:/home  nfs    68510   55804   12706  81%  /usr/share/help

BSD systems use -t option to specify the fstype. The output from the df command also varies among the different operating systems. df also resolves the symbolic links and determines the file system mounted at the link's target. For example:

     # ls -l /usr/local/man
     lrwxr-xr-x    1 root     sys            6 Mar 17  1995 /usr/local/man -> catman/
     # df /usr/local/man
     Filesystem              Type  blocks     use   avail %use  Mounted on
     orange:/usr/local        nfs   68510   55804   12706  81%  /usr/local

Use the command nfsstat -s to display NFS activity on the server side. For example:

     # nfsstat -s

     Server RPC:
     calls      badcalls   nullrecv   badlen     xdrcall    duphits    dupage
     50852      0          0          0          0          0          0.00    

     Server NFS:
     calls        badcalls     
     50852        0            
     null         getattr      setattr      root         lookup       readlink  
     1  0%        233  0%      0  0%        0  0%        1041  2%     0  0%  
     read         wrcache      write        create       remove       rename    
     49498 97%    0  0%        0  0%        0  0%        0  0%        0  0% 
     link         symlink      mkdir        rmdir        readdir      fsstat 
     0  0%        0  0%        0  0%        0  0%        75  0%       4  0%

The output may be interpreted using the following guidelines.

badcalls > 0 - RPC requests are being rejected by the server. This could indicate authentication problems caused by having a user in too many groups, attempts to access exported file systems as root, or an improper Secure RPC configuration.
nullrecv > 0 - NFS requests are not arriving fast enough to keep all of the nfsd daemons busy. Reduce the number of NFS server daemons until nullrecv is not incremented.
symlink > 10% - Clients are making excessive use of symbolic links that are on file systems exported by the server. Replace the symbolic link with a directory, and mount both the underlying file system and the link's target on the client.
getattr > 60% - Check for non-default attribute caching (noac mount option) on NFS clients.

On the client side use the command nfsstat -c to display the client statistics. For example:

     # nfsstat -c

     Client RPC:
     calls      badcalls   retrans    badxid     timeout    wait       newcred
     369003     62         1998       43         2053       0          0 

     Client NFS:
     calls        badcalls     nclget       nclsleep     
     368948       0            368948       0            
     null         getattr      setattr      root         lookup       readlink  
     0  0%        51732 14%    680  0%      0  0%        95069 25%    542  0% 
     read         wrcache      write        create       remove       rename 
     210187 56%   0  0%        2259  0%     1117  0%     805  0%      337  0%   
     link         symlink      mkdir        rmdir        readdir      fsstat    
     120  0%      0  0%        7  0%        0  0%        5510  1%     583  0%

This output may be interpreted using the guidelines given below.

timeout > 5% - The client's RPC requests are timing out before the server can answer them, or the requests are not reaching the server. Check badxid to determine the problem.
badxid ~ timeout - RPC requests are being handled by the server, but too slowly. Increase timeo parameter value for this mount, or tune the server to reduce the average request service time.
badxid ~ 0 - With timeouts greater than 3%, this indicates that packets to and from the server are getting lost on the network. Reduce the read and write block sizes (mount parameters rsize and wsize) for this mount.
badxid > 0 - RPC calls on soft-mounted file systems are timing out. If the server is running, and badcalls is growing, then soft mounted file systems should use a larger timeo or retrans value.

The major NFS daemons are:

2013-09-04T12:01:00.001-07:00

nfsd: nfsd handles client requests from remote systems. Multiple copies of this daemon are usually run so that several requests can be handled simultaneously. However, too many copies of nfsd can increase the demand for CPU time to the point where a drop in performance results. For the best performance the number of copies of nfsd should be set to four.
biod: biod handles block I/O requests for NFS client processes. As with nfsd, several copies are usually run and the number of copies should be set to four.
rpc.mountd: rpc.mountd handles mount requests from remote systems.
rpc.lockd: rpc.lockd manages file locking on NFS client and server machines.
rpc.statd: rpc.statd manages lock crash and recovery services for both client and server systems.
portmap: portmap is not strictly an NFS daemon, although it is required for NFS to function properly. It facilitates the initial connection between local and remote servers. Under Solaris the rpcbind daemon performs the same function.

Installing NFS is fairly simple. Once the software has been installed and NFS capabilities are enabled in the kernel, the daemons need to be started. This can be done with with a script. Some flavors of Unix start these daemons when NFS is installed or upon a reboot following installation of the NFS software. The daemons can also be started from the command line:
# /usr/sbin/biod 4 # /usr/sbin/nfsd 4 # /usr/sbin/rpc.mountd # /usr/sbin/rpc.statd # /usr/sbin/rpc.lockd

LUN Misalignments and poor I/O performance in real and virtual worlds

2013-09-04T10:40:00.002-07:00

http://www.perfluent.com/1/post/2012/10/lun-misalignments-and-poor-io-performance-in-real-and-virtual-worlds.html

NetApp cabling and hardware basics

2013-08-28T08:52:00.004-07:00

http://blog.hernanjlarrea.com.ar/index.php/netapp-cabling-and-hardware-basics/

Understanding SCSI, ATA, SAS and SATA

2013-08-28T08:50:00.003-07:00

http://www.webopedia.com/DidYouKnow/Computer_Science/2007/sas_sata.asp

Error: There are currently no logon servers available to service the logon request

2012-12-02T05:33:00.000-08:00

CIFS share not accessible on windows server.

"Error: There are currently no logon servers available to service the logon request"

Cause:
There is a problem with Kerberos ticketing breaking the secure channel trust between the storage system and the domain.

Solution:
Perform the following steps to resolve the issue:

Note: These steps are disruptive and will terminate all CIFS sessions.

    [root@myadminhost ~]# rsh myfiler cifs resetdc ad1.mywindomain.com (domain name)

Then check access for share...

[root@myadminshost1 ~]# rsh myfiler cifs shares

Name         Mount Point                       Description
----         -----------                       -----------
ETC$         /etc                              Remote Administration
                        BUILTIN\Administrators / Full Control
HOME         /vol/vol0/home                    Default Share
                        everyone / Full Control
C$           /                                 Remote Administration
                        BUILTIN\Administrators / Full Control

myshare     /vol/my_vol/my     crmod_sldc
                        AD\Domain Users / Full Control

Note:
if this doesn't help then cifs terminate and then run cifs setup and add Domain controller again.

NFS Server Not Responding

2012-07-05T07:53:00.003-07:00

Jun 16 22:14:16 mytesthost kernel: nfs: server mytestfiler-nas OK

Jun 16 22:15:12 mytesthost kernel: nfs: server mytestfiler-nas not responding, still trying

Jun 16 22:15:30 mytesthost last message repeated 4 times

Jun 16 22:15:35 mytesthost kernel: nfs: server mytestfiler-nas OK

If above Error comes then please check host health like lod, iostat, free/used/swap memory and number of processes on host for that time. Also check if any network saturation occurred or packet loss happened at that time when message came.

SnapMirror and SnapVault PUSH or PULL ?

2012-06-17T08:39:00.000-07:00

SnapMirror and SnapVault

SnapMirror and SnapVault use TCP port 10566 for data transfer. Network connections are always initiated by the destination system; that is, SnapMirror and SnapVault pull data rather than push data.

Authentication is minimal with both SnapMirror and SnapVault. To restrict inbound TCP connections on port 10566 to a list of authorized hosts or IP addresses, you should configure the snapmirror.access or snapvault.access option. When a connection is established, the destination storage system communicates its host name to the source storage system, which then uses this host name to determine if a transfer is allowed. You should confirm a match between the host name and its IP address. To confirm that the host name and the IP address match, you should set the snapmirror.checkip.enable option to on.
To disable SnapMirror, you should set the snapmirror.enable option to off. To disable SnapVault, you should set the snapvault.enable option to off.

Disk Statistics

2012-06-14T02:22:00.000-07:00

# priv set "advanced;disk shm_stats"

Warning: These advanced commands are potentially dangerous; use
         them only when directed to do so by NetApp
         personnel.
                        Disk   Average    Max    Retry Timeout Sense Data
Disk                    State    I/O      I/O    count count    1      2      3     4     5     9     B     Other
------------------------------------------------------------------------------------------------------------------
1a.10.0                   007      000    0839    000    000    0000    000    000    00    00    00    00    000
1a.10.1                   007      000    0854    000    000    0000    000    000    00    00    00    00    000
1a.10.2                   007      000    1262    000    000    0000    000    000    00    00    00    00    000
1a.10.3                   007      000    0855    000    000    0000    000    000    00    00    00    00    000
1a.10.4                   007      000    0846    000    000    0000    000    000    00    00    00    00    000
1a.10.5                   007      000    0874    000    000    0000    000    000    00    00    00    00    000
1a.10.6                   007      000    0856    000    000    0000    000    000    00    00    00    00    000
1a.10.7                   007      000    0838    000    000    0000    000    000    00    00    00    00    000
1a.10.8                   007      000    0824    000    000    0000    000    000    00    00

# priv set "advanced;disk_stat 1b.00.18"

Warning: These advanced commands are potentially dangerous; use
         them only when directed to do so by NetApp
         personnel.
1b.00.18: ??? HPrio=0 out=0 queued=0 (14346/0/0)
      max_Q=1 max_b=1 recovered error=0 medium error=0
      not ready count=0 command timeout count=0
disk queue histogram:
   4790      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
Message history:
    DISK_READ:                   12
    DISK_READ_CHAIN:             4778
    DISK_WRITE:                  0
    DISK_WRITE_CHAIN:            0
    DISK_WRITE_CHAIN_VERIFY:     0
    DISK_READ_WITH_CKSUM:        0
    DISK_READ_CHAIN_WITH_CKSUM: 0

How to Create and Acces ISCSI LUN

2012-06-02T03:43:00.000-07:00

http://www.howtoforge.com/comment/reply/2459

NetApp Disk Administration

2012-05-29T20:59:00.002-07:00

http://www.datadisk.co.uk/html_docs/netapp/netapp_disk.htm

In this section it will cover the basics on the following:

Storage
Disks
Aggregates (RAID options)
Volumes (FlexVol and Traditional)
FlexCache
FlexClone
Deduplication
QTrees
CIFS Oplocks
Security styles
Quotas

How To Add Shelf in Netapp Filer

2012-05-27T06:14:00.001-07:00

To add SAS shelf,

1.power on the shelf and set the shelf id and power off

2.we can add online, to the sas adapter port. (cluster we need to connect as a stack(multipath), so that each filer will see all the shelf)

3. once you connected just power on

4. 'sasadmin shelf' command to see the shelf on filer.

To add FC shelf,

1.power on the shelf and set the shelf id and power off

2.configure the chosen fc port (0a/0b) as a initiator (fcadmin config -t initiator 0a)

3.then reboot the filer to take effect, once filer comes up just power on the fc shelf

4. 'fcadmin device_map' command to see the shelf on the filer.

Upgrading Netapp Filer

2012-05-27T05:50:00.003-07:00

Upgrading Data ONTAP vary depending on the model of NetApp controller and the version of ONTAP and System Disk, Shelf and RLM firmware currently running on the system. This procedure attempts to address all supported models and configurations. In summary, everything falls into one of the following:

- Models: FAS9x0, FAS3050, FAS3070, FAS3170, FAS60x0

- Data ONTAP: is some variant of a 7.2x or 7.3x release

The goal of this procedure is to provide steps for upgrading to:

- Data ONTAP: v7.3.6P2

- System Firmware: FAS9x0=v4.3.1, FAS3050=v3.1, FAS3070=v2.5, FAS3170=v4.4, FAS32x0=v5.1.1, FAS60x0=v1.9

- Shelf Modules: ESH2=v20, ESH4=v14, AT-FCX=v38, IOM3=v131

- RLM: v4.0 (requires upgrade to v3.1 or v3.1p1 first before upgrading to v4.0)

- Disks: (various, is best to install the current all.zip to get the latest of all disk firmware)

It has been confirmed that the Data ONTAP 736p2 installation package files contains all the latest System Firmware and Shelf Module versions listed above. This reduces the number of files which must be downloaded to three at most (ONTAP, Disk and RLM). NOTE: One exception to this is FAS32x0 systems whose latest System Firmware is 5.1.1 and the 736p2 kit includes only v5.1. For FAS32x0 systems, (amend this procedure to download and update the flash card with 5.1.1. before the reboot.

________________________________________

BEFORE THE OUTAGE WINDOW:

Note: confirm aggregates have 3-5% free:

On each head: > df -Ah

Downloading software from netApp NOW site to /etc/software:

On each head: pre-load the following files into vol0:/etc/software

• Data ONTAP (one of):

736P2_setup_e.exe - pc_elf - use for FAS3020 or FAS3050 only

736P2_setup_i.exe - intel - use for FAS960 or FAS980 only

736P2_setup_q.exe - x86-64 - use for FAS3070, FAS3140, FAS3170, FAS6070, FAS6080 only

• all.zip ... contains all disk fw – Is not needed if recently upgraded and no updates since but can't hurt to install latest all.zip

• RLM_FW.zip - 3.1.0P1 – not needed if FAS980 or if already running RLM 4.0 - download and name file: RLM_FW_3.1.0P1.zip (1)

• RLM_FW.zip - 4.0.0 – not needed if FAS980 or if already running RLM 4.0 - download and name file: RLM_FW_4.0.0.zip (1)

NOW site URL links for above files:

• Data ONTAP (depending on model per above): NOTE: you may have to cut/paste these links into your browser

http://now.netapp.com/NOW/download/software/ontap/7.3.6P2/pc_elf/736P2_setup_e.exe

http://now.netapp.com/NOW/download/software/ontap/7.3.6P2/intel/736P2_setup_i.exe

http://now.netapp.com/NOW/download/software/ontap/7.3.6P2/x86-64/736P2_setup_q.exe

• Checksums for Data ONTAP files (should check this after the file resides in vol0/etc/software):

736P2_setup_e.exe - 61d719469ede61a9c7ac6acbeff4fced

736P2_setup_i.exe - bf7346995b77dea54075fce47b002437

736P2_setup_q.exe - 3992396d9c60468137ebafbc078c7371

• Disk and RLM Firmware: NOTE: you may have to cut/paste these links into your browser

http://now.netapp.com/NOW/cgi-bin/diskfwmustread.cgi/download/tools/diskfw/bin/all … scroll to bottom [Download .zip]

http://now.netapp.com/NOW/cgi-bin/rlmblic.cgi/download/tools/rlm_fw/4.0.0/ontap_cli.shtml

http://now.netapp.com/NOW/cgi-bin/rlmblic.cgi/download/tools/rlm_fw/3.1.0P1/ontap_cli.shtml … see NOTE(1) above

________________________________________

NOTE(1): RLM 4.0 is the goal and requires upgrading to RLM 3.1.0P1 first, so both 3.1.0p1 and 4.0 versions may be needed

________________________________________

BEFORE THE OUTAGE WINDOW:

NOTE: These steps can be performed while connected to the controller via ssh/rsh/telnet. Use a terminal tool (i.e.; putty) which records the session to a log file in the event NetApp support assistance is required.

on each head: > options autosupport.doit before-upgrade

On each head: > software update 736P2_setup_x.exe -r ... (where x is e, i, m or q per above) ( TIME: 10 to 30 minutes )

The system prints a series of dots "........" while writing the kernel and diagnostics to the flash card and disks

Use version -b command to confirm that flash card holds 736P2 as the primary kernel

hostname> version -b

1:/x86_elf/kernel/primary.krn: OS 7.3.6P2 <-- correct

...

OPTIONAL: IF any aggregates on the system are RAID4 THEN do the following:

On each head: > software install all.zip (This ensures the latest disk FW is on the system during the halt/boot process. During boot the systems will upgrade any RAID4 aggregates so this ensures the latest version is unpacked at boot time)

OPTIONAL: IF RLM is not at v3.1 or v3.1P1 (see 'rlm status') THEN do the following:

On each head: > software install RLM_FW_3.1.0P1.zip

On each head: > rlm update ... ( TIME: 20 to 30 minutes )

On each head: Do you want to reboot the RLM now? (y/n) :? y

On each head: > rlm status ... check/repeat that it has rebooted to Online and is running v3.1p1

OPTIONAL: IF RLM is not at v4.0 (see 'rlm status') THEN do the following:

On each head: > software install RLM_FW_4.0.0.zip

On each head: > rlm update ... ( TIME: 20 to 30 minutes )

On each head: Do you want to reboot the RLM now? (y/n) :? y

On each head: > rlm status ... check/repeat that it has rebooted to Online and is running v4.0

________________________________________

DURING OUTAGE WINDOW:

NOTE: These steps should only be performed while connected to the serial console (directly or via term server) or via an RLM ssh terminal connection. Use a terminal tool (i.e.; putty) which records the session to a log file in the event NetApp support assistance is required.

OPTIONAL: IF AT-FCX Shelf Module firmware needs upgraded THEN do that now (see steps above) ... ( TIME: 10 to 20 minutes )

On one head: > cf disable

On each head: > halt -f

OPTIONAL: IF not a FAS9x0 THEN this step is likely needed (all FAS9x0 are all already up-to-date)

On each head: at the Ok>, CFE>, -or- LOADER>

prompt > update_flash

The system prints a series of =+=+=+=+=+=+=+=+=+=+=+=+=+=+ while updating system firmware

On each head: at Ok/CFE/LOADER prompt > bye

On each head: > watch boot and look for errors

OPTIONAL: IF prompts about "low NVRAM battery" appear THEN override by ctrl-C and y:

WARNING: The battery voltage is too low to hold data ...etc...

To override this delay, press CTRL-C. <control-C>

CAUTION: Using this appliance without NVRAM ...etc...

Are you sure you want to continue (y or n)? y

OPTIONAL: IF shelf modules are still down-rev THEN one head may stall while booting up and update the shelves:

<d>[<h>: sfu.controllerElementsPerShelf:info]: [storage download shelf]: 2 ES controller elements can be updated on 0b.shelf1

<d>[<h>: sfu.controllerElementsPerShelf:info]: [storage download shelf]: 2 ES controller elements can be updated on 0b.shelf2

<d>[<h>: sfu.controllerElementsPerShelf:info]: [storage download shelf]: 2 ES controller elements can be updated on 0b.shelf3

...

On each head: <login with admin user/password>

On each head: > version ... (to confirm version of Data ONTAP is 7.3.4P4)

On one head: > cf enable ... (to re-enable cluster failover)

On each head: > aggr status ; vol status ... (to confirm everything is online)

OPTIONAL: IF shelf firmware is still not up-to-date for some reason (check with sysconfig -a) THEN perform:

On one head: > priv set advanced

On one head: *> storage download shelf ... ( TIME: 10 to 20 minutes )

<d>[<h>: sfu.controllerElementsPerShelf:info]: [storage download shelf]: 2 ES controller elements can be updated on 0b.shelf1

<d>[<h>: sfu.controllerElementsPerShelf:info]: [storage download shelf]: 2 ES controller elements can be updated on 0b.shelf2

<d>[<h>: sfu.controllerElementsPerShelf:info]: [storage download shelf]: 2 ES controller elements can be updated on 0b.shelf3

...

NOTE: The dual modules in each shelf are downloaded in two waves, each taking about 5 to 10 minutes

Ends with the message: <date> [<hostname>: sfu.downloadSummary:info]: Shelf firmware updated on ## shelves.

On one head: *> sysconfig -a ... (look for all Modules to be current ESH2=v20, ESH4=v14, AT-FCX=v38)

For example: ... Look at the end of the list of disks on each loop at the Shelf 1,2,3,4,etc lines for proper version

... NETAPP X275_S15K4146F15 NA01 136.0GB 520B/sect (3KN0AC6J000075505WA8)

93 : NETAPP X275_S15K4146F15 NA01 136.0GB 520B/sect (3KN0CPBB00007601CF6L)

Shelf 1: ESH2 Firmware rev. ESH A: 20 ESH B: 20

Shelf 2: ESH2 Firmware rev. ESH A: 20 ESH B: 20 <-- both modules running v20

Shelf 3: ESH2 Firmware rev. ...

OPTIONAL: IF not all modules were updated THEN do the following on the other head

On one head: > priv set advanced

On one head: *> storage download shelf ... ( TIME: 10 to 20 minutes )

NOTE: The dual modules in each shelf are downloaded in two waves, each taking about 5 to 10 minutes

Ends with the message: <date> [<hostname>: sfu.downloadSummary:info]: Shelf firmware updated on ## shelves.

On one head: > cf status ... (to confirm cluster failover status is enabled, sometimes it takes a few minutes to fully enable)

At this point the disruptive part of the upgrade is over, inform users

________________________________________

AFTER THE OUTAGE WINDOW:

NOTE: These steps can be performed while connected to the controller via ssh/rsh/telnet. Use a tool which records the session to a log file in the event NetApp support assistance is required.

OPTIONAL: IF the system is licensed for CIFS (check via license command) then do the following per CSB-1201-02

On each head: > options cifs.rpcfd.timeout -1

OPTIONAL: IF not done before the upgrade THEN do the following now:

On each head: > software install all.zip (after this, any down-rev disks may begin being upgraded in background)

OPTIONAL: IF RLM is not at v3.1 or v3.1P1 (see 'rlm status') THEN do the following:

On each head: > software install RLM_FW_3.1.0P1.zip

On each head: > rlm update ... ( TIME: 20 to 30 minutes )

On each head: Do you want to reboot the RLM now? (y/n) :? y

On each head: > rlm status ... check/repeat that it has rebooted to Online and is running v3.1p1

OPTIONAL: IF RLM is not at v4.0 (see 'rlm status') THEN do the following:

On each head: > software install RLM_FW_4.0.0.zip

On each head: > rlm update ... ( TIME: 20 to 30 minutes )

On each head: Do you want to reboot the RLM now? (y/n) :? y

On each head: > rlm status ... check/repeat that it has rebooted to Online and is running v4.0

OPTIONAL: IF there is no backup kernel on flash ( check with version -b as shown below ) ( per TSB-1011-01 )

hostname> version -b

1:/x86_elf/kernel/primary.krn: OS 7.3.4P4

1:/backup/x86_elf/kernel/primary.krn: OS 7.3.3P5 <-- backup kernel exists so re-doing 'sw update' cmd is NOT required

1:/x86_elf/diag/diag.krn: 5.4.6 ...

On each head: > software update 734P4_setup_e.exe -r ... ( TIME: 10 to 30 minutes )

On each head: > options autosupport.doit after-upgrade

Ports Used In Netapp Filers

2012-05-26T09:21:00.001-07:00

testfiler> rdfile /vol/vol0/etc/services
rdfile /vol/vol0/etc/services
#
# Network services, Internet style
#
ftp-data        20/tcp
ftp             21/tcp
ssh             22/tcp
telnet          23/tcp
smtp            25/tcp
time            37/tcp           # Time Service
time            37/udp           # Time Service
domain          53/udp           # DNS
domain          53/tcp           # DNS
portmap         111/udp
portmap         111/tcp
dhcps           67/udp           # DHCP server
dhcpc           68/udp           # DHCP client
tftp            69/udp
http            80/tcp
kerberos        88/udp           # Kerberos 5
kerberos        88/tcp           # Kerberos 5
nntp            119/tcp
ntp             123/tcp          # Network Time Protocol
ntp             123/udp          # Network Time Protocol
netbios-name    137/udp          # NetBIOS nameserver
netbios-dg      138/udp          # NetBIOS datagram service
netbios-ssn     139/tcp          # NetBIOS service session
snmp            161/udp
ldap            389/tcp          # LDAP session
https           443/tcp          # SecureAdmin/SSL
cifs-tcp        445/tcp          # CIFS over TCP with NetBIOS framing
kpasswd         464/tcp          # Filer does not listen on this port;
                                 # used as Domain Controller destination port
                                 # for Kerberos passwd set/change operations
shell           514/tcp
syslog          514/udp
route           520/udp
ldap-ssl        636/tcp          # LDAP over SSL
kerberos-sec    750/udp          # For compatibility with older "750" clients
kerberos-sec    750/tcp          # For compatibility with older "750" clients
nfsd            2049/udp
nfsd            2049/tcp
nrv             2050/tcp         # NetApp Remote Volume protocol, used in
                                 # FlexCache and Restore-On-Demand.

iscsi-target    3260/tcp

nlockmgr        4045/tcp         # NLM
nlockmgr        4045/udp
mountd          4046/tcp         # NFS mountd protocol
mountd          4046/udp
status          4047/tcp
status          4047/udp
pcnfsd          4048/tcp         # PCNFS protocol
pcnfsd          4048/udp
rquotad         4049/udp

ndmp           10000/tcp
sm-ics         10565/tcp         # Snapmirror Multipath
snapmirror     10566/tcp
sm-sync-block 10567/tcp         # Snapmirror Sync Block Data
sm-sync-trans 10568/tcp         # Snapmirror Sync Transaction Data
sm-sync-ctrl   10569/tcp         # Snapmirror Sync Control Data
nbu-nearstore 10571/tcp         # NetBackup - Nearstore
sm-ics-test    10670/tcp         # INTERNAL USE: Snapmirror Multipath Test
ndmp-local     32243/tcp         # Internal connection inside NetApp box

How to look Busy Disk on Filer...

2012-05-26T05:45:00.000-07:00

[root@adminhost ~]# rsh <filername> stats show -r -i 2 disk:*:disk_busy |grep -i 100%
disk:8B89FF35:5244805C:310DF900:835F2F:00000000:00000000:00000000:00000000:00000000:00000000:disk_busy:100%
disk:7BAD77C:D43601:7B76BD:F8C4DD:00000000:00000000:00000000:00000000:00000000:00000000:disk_busy:100%
disk:62EB211:E871C0D8:2F631C:708245:00000000:00000000:00000000:00000000:00000000:00000000:disk_busy:100%
disk:B6AA3EB:31F9CC:CA1271:A45EEB:00000000:00000000:00000000:00000000:00000000:00000000:disk_busy:100%
disk:AB114F:C9E624:186E22:D84AB0:00000000:00000000:00000000:00000000:00000000:00000000:disk_busy:100%
disk:7A977C:D43601:7B76BD:F8E4DD:00000000:00000000:00000000:00000000:00000000:00000000:disk_busy:100%
disk:8CA1DB:D55170:8A7025:390EF1C8:00000000:00000000:00000000:00000000:00000000:00000000:disk_busy:100%

How to Run Perfstat ....

2012-05-26T05:16:00.003-07:00

If we need to run perfstat for "5 minutes interval for 2 hours" then run below command

perfstat.sh -F -f <filername> -t 5 -i 24 >> output_file

Disk Shelves and Modules

2012-05-25T04:17:00.000-07:00

Disk type:

ATA Disks: well… you must be already used with these types of disks.
SATA Disks: Also for this one, already well known.
FC disks: Fiber channel disks.

Check URL : https://library.netapp.com/ecmdocs/ECMM1278264/html/smg/provisioning/reference/r_oc_prov_disk-cap.html

Shelf Models

ds14mk2: This shelf accepts 14 FC or ATA disks, it takes 3 rack units and accepts ESH2 and AT-FCX disks.
ds14mk4: This shelf accepts 14 FC disks, it takes 3 rack units and only accepts ESH4 disks.

Modules

Modules allow different disk shelves to be connected to the storage FC loops.

ESH2: This module is used for FC disks and it is connected to a FC 2GB bus.
ESH4: This module is used for FC disks and it is connected to a FC 4GB bus.
AT-FCX: This module is used for ATA or SATA disks and it is connected to a FC 2GB bus.

2012-05-17T03:03:00.001-07:00

Configuring DFM server for Performance Advisor (PA).

Performance Adviser (PA) is resource intensive and having PA enabled along with other monitoring on single server monitoring close to 200+ controllers on a VM or a low end hardware might slow down OnCommand/DFM response times. If a customer has decided to load balance his OnCommand/DFM server based on functionality and trying to find a way to have PA only enabled and no capacity or space monitoring, continue reading.

Step 1:

Install OnCommand/DFM server and do your usual setting like setting up https, disabling rsh and enabling ssh etc.

Step 2:

Turn off all monitoring interval by doing a list of options and greping for moninterval.

[root@]# for i in `dfm options list | grep -i moninterval| awk '{print $1}'`;do dfm options set $i=off;done
Changed agent monitoring interval to Off.
Error: autosupportMonInterval: off is not a valid interval.
Specify a positive number, optionally followed by a time period
suffix to indicate seconds, minutes, hours, days, or weeks.
Examples: 4m, "2.5 hours", "15 secs", 1week.
Changed backup directory monitoring interval to Off.
Changed config conformance monitoring interval to Off.
Changed cluster failover monitoring interval to Off.
Changed cluster monitoring interval to Off.
Changed CPU monitoring interval to Off.
Changed disk free space monitoring interval to Off.
Changed disk monitoring interval to Off.
Changed dataset conformance monitoring interval to Off.
Changed dataset disaster recovery status monitoring interval to Off.
Changed dataset protection status monitoring interval to Off.
Changed environmental monitoring interval to Off.
Changed fibre channel monitoring interval to Off.
Changed file system monitoring interval to Off.
Changed host RBAC monitoring interval to Off.
Changed interface monitoring interval to Off.
Changed license monitoring interval to Off.
Changed LUN monitoring interval to Off.
Changed NDMP monitoring interval to Off.
Changed operation count monitoring interval to Off.
Changed performance advisor threshold violation monitoring interval to Off.
Changed ping monitoring interval to Off.
Changed qtree monitoring interval to Off.
Changed resource pool space monitoring interval to Off.
Changed SAN host monitoring interval to Off.
Changed share monitoring interval to Off.
Changed SnapMirror monitoring interval to Off.
Changed snapshot monitoring interval to Off.
Changed SnapVault monitoring interval to Off.
Changed SRM host monitoring interval to Off.
Changed global status monitoring interval to Off.
Changed system information monitoring interval to Off.
Changed user quota monitoring interval to Off.
Changed vFiler monitoring interval to Off.
Changed virtual server monitoring interval to Off.

STEP 3:

Enable only those monitors that are required for PA functionality to work

[root@]# dfm option set pingMonInterval=
Changed ping monitoring interval to 1 minute.

PingMon is required otherwise OnCommand/DFM does not monitor a host as it considers them to be down.

[root@]# dfm option set diskMonInterval=
Changed disk monitoring interval to 4 hours.

Disk Mon is required for discovery of disk so that performance data for disk can be collected and show in NMC.

[root@]# dfm option set fsMonInterval=
Changed file system monitoring interval to 15 minutes.

FS Mon is required for discovery of volume, qtree, aggr so that corresponding performance data can be collected and show in NMC.

[root@]# dfm option set ifMonInterval=
Changed interface monitoring interval to 15 minutes.

IF Mon is required for discovery of Interfaces so that corresponding performance data can be collected and show in NMC.

[root@]# dfm option set licenseMonInterval=
Changed license monitoring interval to 4 hours.

Though its weird, license Mon is required for discovery of some objects.

[root@]# dfm option set lunMonInterval=
Changed LUN monitoring interval to 30 minutes.

Lun Mon is required for discovery of LUN so that corresponding performance data can be collected and show in NMC

[root@]# dfm option set perfAdvThreshViolationMonInterval=15m
Changed performance advisor threshold violation monitoring interval to 15 minutes.

PerfAdvThresholdViolation Mon is required for PA threshold generation

[root@]# dfm option set vFilerMonInterval=
Changed vFiler monitoring interval to 1 hour.
[root@]#

vFiler Mon is required for discovery of vFiler so that corresponding performance data can be collected and show in NMC

STEP 4:

Check for the list of MonIntervals that are enabled, using the below command.

Windows world run the command as follows (dfm options list | findstr /i moniterval)

[root@]# dfm options list | grep -i moninterval
agentMonInterval                      Off
autosupportMonInterval                2 minutes
backupDirMonInterval                  Off
ccMonInterval                         Off
cfMonInterval                         Off
clusterMonInterval                    Off
cpuMonInterval                        Off
dfMonInterval                         Off
diskMonInterval                       4 hours
dsConformanceMonInterval              Off
dsDRMonInterval                       Off
dsProtectionMonInterval               Off
envMonInterval                        Off
fcMonInterval                         Off
fsMonInterval                         15 minutes
hostRBACMonInterval                   Off
ifMonInterval                         15 minutes
licenseMonInterval                    4 hours
lunMonInterval                        30 minutes
ndmpMonInterval                       Off
opsMonInterval                        Off
perfAdvThreshViolationMonInterval     15 minutes
pingMonInterval                       1 minute
qtreeMonInterval                      Off
respoolSpaceMonInterval               Off
SANHostMonInterval                    Off
shareMonInterval                      Off
snapmirrorMonInterval                 Off
snapshotMonInterval                   Off
snapvaultMonInterval                  Off
srmMonInterval                        Off
statusMonInterval                     Off
sysInfoMonInterval                    Off
userQuotaMonInterval                  Off
vFilerMonInterval                     1 hour
vserverMonInterval                    Off
[root@]#

STEP 5:
Optional, if required disable auto-discover options in by disabling all the below listed discovery options in BOLD

[root@]# dfm options list |grep -i discover
discoverAgents                        Enabled
discoverClusters                      Enabled
discoverEnabled                       Enabled
discoverHostInitEnabled               Enabled
discoverHosts                         Enabled
discoverInterval                      15 minutes
discoverNetworks                      Disabled
discoverSAN                           Enabled
discoverTimeout                       5 seconds
discoverVfilers                       Enabled
hostEnclosureDiscoveryEvents          Disabled
networkDiscoveryLimit                 15
snapshotDiscoveryEventsEnabled        No
[root@]#

STEP 6:

Now add your controllers/filer and set the credentials for API access so that OnCommand/DFM server can start collecting only performance data.

https://communities.netapp.com/docs/DOC-14077

2012-05-17T02:59:00.001-07:00

Click Below Link to see ONTAB Booting Steps

ONTAP 7.3 Boot Menu Options Flowchart

2012-05-16T05:05:00.002-07:00

How to Configure CIFS Shares
+++++++++++++++++++++++++

myfiler> cifs setup

This process will enable CIFS access to the filer from a Windows(R) system.
Use "?" for help at any prompt and Ctrl-C to exit without committing changes.

Your filer does not have WINS configured and is visible only to
clients on the same subnet.

Do you want to make the system visible via WINS? [n]:

A filer can be configured for multiprotocol access, or as an NTFS-only
filer. Since multiple protocols are currently licensed on this filer,
we recommend that you configure this filer as a multiprotocol filer

(1) Multiprotocol filer
(2) NTFS-only filer

Selection (1-2)? [1]:

CIFS requires local /etc/passwd and /etc/group files and default files
will be created. The default passwd file contains entries for 'root',
'pcuser', and 'nobody'.

Enter the password for the root user []:

Retype the password:

The default name for this CIFS server is 'myfiler'.

Would you like to change this name? [n]:

Data ONTAP CIFS services support four styles of user authentication.
Choose the one from the list below that best suits your situation.

(1) Active Directory domain authentication (Active Directory domains only)
(2) Windows NT 4 domain authentication (Windows NT or Active Directory domains)
(3) Windows Workgroup authentication using the filer's local user accounts
(4) /etc/passwd and/or NIS/LDAP authentication

Selection (1-4)? [1]: 1

What is the name of the Active Directory domain? [ad1-WIN.COM]: ad1-win.com

In Active Directory-based domains, it is essential that the filer's
time match the domain's internal time so that the Kerberos-based
authentication system works correctly. If the time difference between
the filer and the domain controllers is more than 5 minutes,
authentication will fail. Time services are currently not configured
on this filer.

Would you like to configure time services? [y]: n

In order to create an Active Directory machine account for the filer,
you must supply the name and password of a Windows account with
sufficient privileges to add computers to the AD1 -WIN.COM
domain.
Enter the name of the Windows user [Administrator@AD1 -WIN.COM]: user@ad1 -win.com ---- Following steps can be done with the help of Windows team.

Password for user@ad1 -win.com: xxxxxxx

CIFS - Logged in as user@ad1 -win.com.
Setup was unable to retrieve a list of joinable containers
(organizational units) from Active Directory, therefore a list of
selectable options cannot be provided. Please enter the distinguished
name of the container that you would like the filer to join below.
There is no need to add the domain name portion,
'DC=ad1,DC=odss,DC=ora-win,DC=com', of the distinguished name.
Enter the name of the organizational unit [CN=Computers]:
CIFS - Starting SMB protocol...
It is highly recommended that you create the local administrator
account (myfiler\administrator) for this filer. This account allows
access to CIFS from Windows when domain controllers are not
accessible.

Do you want to create the myfiler\administrator account? [y]: n

Welcome to the AD1 -WIN.COM (AD1) Active Directory(R) domain.

CIFS local server is running.
myfiler>

Now Add CIFS shares ---

Create CIFS share with Below Command

[root@myhost ~]# ssh myfiler cifs shares -add myshare /vol/my_volume/

Add comment to New share

[root@ myhost ~]# ssh myfiler cifs shares -change myshare -comment myshare_imp

1 share(s) have been successfully modified

List Shares

[root@ myhost ~]# rsh myfiler cifs shares

Name Mount Point Description

---- ----------- -----------

CRMdepot /vol/my_volume/ crmod_sldc

everyone / Full Control

Enable below Options on Volume to give better performance

convert_ucode=on

create_ucode=on

Also, enable below option in Multiprotocol Filer

[root@ myhost ~]# rsh myfiler options |grep -i syml|grep enable

cifs.symlinks.enable on

Now try Accessing your share from win server for which u added in CIFS setup

2012-05-06T09:12:00.002-07:00

Replacing a Failed Disk on Netapp

http://geekdom.wesmo.com/2007/01/30/replacing-a-failed-disk-on-a-network-appliance-netapp/

2012-05-06T09:10:00.003-07:00

How to Solve Failabck/Giveback Errors in Netapp.....

https://communities.netapp.com/servlet/JiveServlet/previewBody/5352-102-1-7563/PS%20Tech%20Note%20053%20-%20Solve%20cluster%20failover%20Issues.pdf

Netapp Options checklist, Very useful ...

2012-03-17T06:45:00.002-07:00

http://netapp.uptime.be/free/NetApp_Post_Installation_Checklist.html

How to List FC Luns/SAN from Linux Hosts....

2011-12-16T06:30:00.000-08:00

myhost# sanlun lun show -p --> to list all luns on host with mapper names...

myhost# ls -l /dev/mapper ---> to list mappers ( chk /etc/fstab which you have mounted...

for detailed info also refer "http://technologist.pro/storage/multipathing-netapp-lun-on-rhel-5-3"

2011-11-17T05:01:00.000-08:00

Lun Resizing on Netapp Filer & Mount on Windows Host

Step 1: first offline the lun
2. then run the command lun resize <lun path> newsize
3. if existing size is 10g and you want to add 5g to it. give 15g as the new size
4. online the lun. then from the windows host, run diskmgmt.msc
5. it will show that the new space as unallocated space

6. run the command diskpart from run prompt ( On windows Host)
7. list volume
8. select <volume>
9. extend
that's it
10. verify using disk managemnet that the new size is taken and you are able to open that partition
-------------------------------------------------------------------------------------------