Server

How to restart Jiffy Server ?

Do the following to stop and start Jiffy application:

• Jiffy:
  1. Log in to the application server as a jiffyapp-usr (application user).
  2. Run the following commands:
     • Stop the application:
       application stop all
     • Start the application:
       application start all
     • Select the Master key method from the following options:
       Choose Master key method:
       1. Masterkey from file: Type the absolute file path from which the key must be read.
          
          
       2. Masterkey from as user: Type the user's own Passphrase.
          
          
       Note: Master Key allows password encryption and decryption. User is provided with an option to choose the default Master Key or user's own Master Key.
• Expression:
  1. Log in to the application server as a jiffyexp-usr (jiffy expression user).
  2. Run the following commands:
     • Stop the expression:
       ps -ef | grep expression | grep -v "grep expression"| awk '{ print $2 }'| xargs kill -9
     • Start the expression:
       $JIFFY_HOME/expression/start_expression.sh
• Vault:
  1. Log in to the application server as a iffyapp-usr (application user).
  2. Run the following commands:
     • Stop the vault:
       ps -ef | grep vault | grep -v "grep vault"| awk '{ print $2 }'| xargs kill -9
     • Start the vault launcher:
       $JIFFY_HOME/.vault.d/vault_launcher.sh
     • Unseal the vault:
       $JIFFY_HOME/.vault.d/vault_unseal.sh
• If PostgreSQL DB is installed in the jiffy server, do the following to restart PostgreSQL
  1. Log in to the application server as a root user.
  2. Run the following commands:
     • Stop PostgreSQL DB:
       service postgresql-9.6 stop
     • Create a directory in /var/run/:
       mkdir /var/run/postgresql/
     • Change ownership to Postgres:
       chown -R postgres:postgres /var/run/postgresql/
     • Start PostgreSQL DB:
       service postgresql-9.6 start

What to do if you get DB Connectivity issue/Connection refused on DB port 5432?

Version: This solution is applicable for all versions of Jiffy.

Pre-requisites: putty, root access

Steps to solve:

• You will be seeing an error message like “Connection refused” for port 5432 as below:

• Login to putty, as 'root' user and check the postgres database status.
systemctl status postgresql-9.6
• You will see the status as 'inactive'.


• To start the postgres database run below command:
systemctl start postgresql-9.6
• Next, check the status and it should show as “active”.


• Now login to Jiffy page and it should be up and running fine.
Note: In some case Jiffy server also might be down. So please refer “FAQ 1 : How to stop and start Jiffy server?” And follow the steps to bring it up.

How to create a new Jiffy Admin user??

Version: This solution is applicable for all versions of Jiffy.

Pre-requisites: Putty, Jiffy Server jiffyapp-usr credentials.

Steps to do:

• Open putty and login to Jiffy server.
• Change the user to jiffyapp-usr and go to JIFFY_HOME folder.
su jiffyapp-usr

• Note: JIFFY_HOME -- usually it will be in /opt/jiffy folder. However, this differs from client-to-client and depends on the location where jiffy is installed.

• Now activate the jiffy environment
source /opt/jiffy/.envs/jiffyEnv/bin/activate

• Change the directory to the src folder
(jiffyEnv) [root@localhost objects]# cd /opt/jiffy/deploy/jiffy/src/
• (jiffyEnv) [root@localhost src]#

• Run the command python manage.py setup_admin and provide all the details.
(jiffyEnv) [root@localhost src]# python manage.py setup_admin

Please enter Username:
Please enter Password:
Please confirm the Password:
Please enter Email:
Please enter the first name of the user:
Please enter the last name of the user:
Creating the admin user...
User is created with timezone as UTC.
Please update to correct user timezone by logging.

How to do new Python library installation for expression node?

Version: This solution is applicable for all versions of Jiffy.

Pre-requisites: Enough disk space to download the files.

Note: Before writing any python expression using expression/web node you have to install all python packages required by that expression in the expression environment.

Steps to install Python Packages in Windows Environment (Web Node)
1. Download all pip packages required for expression from [ https://pypi.python.org/pypi ]

2. Install pip packages at command prompt using the below command
pip install --no-index --find-links=file: package name.
Eg: pip install --no-index --find-links=file:/usr/share/localrepos/pypi-packages/ lxml

Steps to install Python Packages in Linux Environment (Expression Node)
1. Download all pip packages required for expression from [ https://pypi.python.org/pypi ]

2. Login to jiffy application server and copy all pip packages required for expression to server location /var/tmp/ using Winscp or JAM.

3. Switch to jiffy expression user using the su command
For example : su jiffyexp-usr

4. Activate jiffy expression environment using the below command
source /jiffy/expression//bin/activate
Eg: source /opt/jiffy/expression/jiffyexpenv/bin/activate

5. Install python packages using the below steps
pip install --no-index --find-links=file: pip package name
Eg : pip install --no-index --find-links=file:/usr/share/localrepos/pypi-packages/ lxml

How to access and query PostgresSQL(Jiffy) Database??

Version: This solution is applicable for all versions of Jiffy.

Pre-requisites: Putty, Jiffy Server jiffyapp-usr credentials.

Info: Using psql, we can generate a complete list of commands by using the help command and access Jiffy app database and tables.

• Database parameter can be found in "$JIFFY_HOME/deploy/jiffy/src/SterlingTester/local_settings.py".

1. Connect to psql shell by using below command:
[root@localhost ~]# psql jiffyappdb jiffy_db_user [syntax => psql ]


2. To see the relations (tables, sequence) run "d" in the shell


3. To see only relation of type Tables run "dt" in the shell
e.g.


4. To See only relations of type Sequence run "ds" in the shell
e.g.


5. Sample select queries.
e.g.

Where is ANT/Maven plugin JAR available for integration of Jenkins with Jiffy?

Version: This solution is applicable for all versions of Jiffy.

Pre-requisites: Putty, Jiffy Server jiffyapp-usr credentials, sufficient disk space to download the files.

Steps:

• Open putty as 'root' user.

• Jars are available in $JIFFY_HOME/deploy/jiffy/src/bin folder.
[root@localhost bin]# pwd
/opt/jiffy/deploy/jiffy/src/bin
[root@localhost bin]# ls -ltrh
total 196M
-rw-rw-r--. 1 jiffyapp-usr jiffyapp-usr 72M Oct 5 15:50 OReng-1.2.jar
-rw-rw-r--. 1 jiffyapp-usr jiffyapp-usr 12M Oct 5 15:50 QR-1.0.jar
-rw-rw-r--. 1 jiffyapp-usr jiffyapp-usr 26M Oct 5 15:50 jiffy-service-1.9.5.jar
-rw-rw-r--. 1 jiffyapp-usr jiffyapp-usr 32M Oct 5 15:51 jiffy-service-manager-1.1.1.jar
-rw-rw-r--. 1 jiffyapp-usr jiffyapp-usr 15M Oct 5 15:52 jiffy-ant-tasks-1.2.jar
-rw-rw-r--. 1 jiffyapp-usr jiffyapp-usr 14M Oct 5 15:52 jiffy-tester-maven-plugin-1.2.jar
-rw-rw-r--. 1 jiffyapp-usr jiffyapp-usr 28M Oct 5 15:52 document-service-1.0.jar
[root@localhost bin]#

How to start an 'Expression Service' or 'Expression Node' ?

Version: This solution is applicable for all versions of Jiffy.

Pre-requisites: Putty, Jiffy Server jiffyexp-usr credentials.

Steps to do:

• Open putty and login to Jiffy server.
• Change the user to jiffyexp-usr and go to JIFFY_HOME folder.
su jiffyapp-usr

Note: JIFFY_HOME -- usually it will be in /opt/jiffy folder. However, this differs from client-to-client and depends on the location where jiffy is installed.

Example of error message in expression Node --> [ root> JiffyData status="FAIL" message="Error in Manager - Details: Expression service is not available. Please contact JIFFY administrator"/ /root>

• Now go to expression directory:
cd /ebs-volume01/jiffy/expression/
>br> • Next source the activate
source /ebs-volume01/jiffy/expression/Jiffyexpenv/bin/activate(type source and pageup)

• Next export JIFFY_EXPRESSION_LOCATION
export JIFFY_EXPRESSION_LOCATION="/ebs-volume01/jiffy/deploy/jiffy/uiscripts/precompile/expression/"(type export and pageup)

• Now start the expression script.
./start_expression.sh

What to do if you get error “Syntax Error: 'charmap' codec cant decode byte ?

Version: This issue was seen in version 2.7.

Pre-requisites: Admin access in Jiffy application.

Steps to do: If you come across below error message as shown in screenshot please follow the steps as described below.
If still getting any issues after this, please contact jiffy-support@option3.io email id.


• Check the UI app configuration in mojo.
• If the 'Encoding' says 'windows-1252' - it will work fine with the normal character, but if it comes across any special characters like "&" or "@" in any fields , including password, it will throw the error as mentioned in the above screenshot.
• Hence change the app configuration in 'Encoding' mode as 'utf-8'.
• Below screenshot for reference:

What to do if under Task executions are not moving from 'Scheduled' to 'Executing' status ” ?

Version: This solution is applicable for all versions of Jiffy.

Pre-requisites: Putty, Jiffy Server jiffyapp-usr credentials.

Steps to do:

• Open putty and login to Jiffy server.
• Change the user to jiffyapp-usr and go to JIFFY_HOME folder.
su jiffyapp-usr

Note: JIFFY_HOME -- usually it will be in /opt/jiffy folder. However, this differs from client-to-client and depends on the location where jiffy is installed.

• Log in to linux server through putty and check the “supervisorctl status”.
• Everything should be up and running fine.
• Please refer FAQ:1 if you see any of the process having issues.


• If everything is running fine, check the 'Diskspace' in RabbitMQ link:
Please refer FAQ:5 for the steps.

• If diskspace is low in RabbitMQ, then scheduling processes will not run.
• For this we should delete some old log files and make some space.
• Login to putty and go to /var/log/rabbitmq
• Remove huge sized * .gzip log files

• This should solve the issue. But if disk space looks good then check the 'Queues' to see if there are high pending messages to be processed.
• Please refer FAQ:4 for the steps.

• If any of the queues are showing high numbers in 100+ messages, this means one of the jiffy processes is hung or having some issue.
• So try restarting all the processes:
ie., login to putty as jiffyapp-usr.
• Stop all the jiffy processes under user "jiffyapp-user": [jiffyapp-usr@localhost scripts]$ ./stop_all.sh
• . start all the jiffy processes and check the uptime : [jiffyapp-usr@localhost scripts]$ ./start_all.sh jiffy
• Please refer FAQ:1 for detailed steps.

• Now verify the queues in RabbitMQ , it should get cleared.
• If any of the processes are not running then stop and start that service alone.
• Refer FAQ:12 for the steps.

How to check connectivity to a smtp server from Jiffy server and send a sample test email though telnet?

Version: This solution is applicable for all versions of Jiffy.

Pre-requisites: Putty, Jiffy Server jiffyapp-usr credentials.

Steps to do:

• First telnet to the SMTP server.
[jiffyapp-usr@localhost root]$ telnet 10.11.30.54 25
Trying 10.11.30.54...
Connected to 10.11.30.54.

• This validates that connectivity to smpt server from Jiffy server was successful.
• To come out of the session press
Escape character '^]'.

• To send a test email, follow below steps:
220 pexmbx04.dpg.local Microsoft ESMTP MAIL Service ready at Tue, 22 Jan 2019 08 :47:44
+0400 MAIL FROM:
503 5.5.2 Send hello first
503 5.5.2 Send hello first
500 5.3.3 Unrecognized command
ehlo 10.128.22.168
250-pexmbx04.dpg.local Hello [10.11.30.5]
250-SIZE 10485760
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES

250-STARTTLS
250-X-ANONYMOUSTLS
250-AUTH NTLM
250-X-EXPS GSSAPI NTLM
250-8BITMIME
250-BINARYMIME
250-CHUNKING
250-XEXCH50
250-XRDST
250 XSHADOWREQUEST

• Provide the 'From' address:
MAIL FROM:
250 2.1.0 Sender OK
• Provide the 'To' address:
RCPT TO:
250 2.1.5 Recipient OK
• Send the Data field
DATA
354 Start mail input; end with .
• Provide a 'subject'
Subject: test
This is a test message
.
250 2.6.0 <005971ecc6fc4eed9091b5431f545667@pexmbx04.dpg.local> [InternalId=511101110120, Hostname=pexmbx04.dpg.local] Queued mail for delivery
Connection closed by foreign host.
[jiffyapp-usr@localhost root]$ telnet 10.11.30.54 25
Trying 10.11.30.54...
Connected to 10.11.30.54.
Escape character is '^]'.
[jiffyapp-usr@localhost root]$

• Now, check your mail box and you should see an email with subject 'test'.

Unix:///var/tmp/supervisor.sock no such file.

Cause: As per the default settings supervisor.sock, the file is created in /var/tmp directory. As per the RHEL policy, the tmp folder gets deleted after every 30 days. So, it gets deleted if there is no restart for 30 days. When this happens, the error unix:///var/tmp/supervisor.sock no such file” gets triggered while running supervisorctl command.

Solution: Below steps can be followed to change the settings for the path to create the supervisor.sock file.

1. Stop the application as a jiffyapp-usr (application user).
   application stop all
2. Login as a root user and update the file supervisord.conf which is present inside the /etc directory. The http_port and the serverurl parameters should be changed with the new path as shown.
   
3. As a jiffyapp-usr, the File, http_port and serverurl parameters should be changed as below in the jiffy-supervisor_nonroot.conf file. The file is located at $JIFFY_HOME/supervisor.
   
   Instead of /var/tmp/supervisor.sock, update $JIFFY_HOME/supervisor.sock
   Example: /opt/jiffy/supervisor.sock
4. Start the application:
   application start all

Internal Server Error

Cause: This error occurs when there is some issue with Jiffy server's components or the DB connection is broken due to several reasons which can be seen in the Error message or in the server logs.

Solution:

• Validate the DB connectivity using SQL developer or other equivalent Database IDE if Jiffy is pointed to some external Oracle DB server.
• If Jiffy is pointed to PostgreSQL, then validate the connection or check if the process is running.
  Example:
  To See the status: systemctl status postgresql-9.6.service
• For more information from log refer the logs in path '$JIFFY_HOME/logs'
  Example:
cd $JIFFY_HOME/logs
ls -ltrh | tail
• Both the above connection validation can be done from the Jiffy Server.
  Example:
  
  • Login to Jiffy Server using putty or equivalent and switch to 'jiffyapp-usr'
  • cd $JIFFY_HOME/deploy/jiffy/src/
  • source $JIFFY_HOME/.envs/jiffyEnv/bin/activate
  • To validate DB connection: python manage.py dbshell
• If Oracle DB connection is down, contact the Oracle DBA/Team and confirm.
• In case of PostgreSQL start, it if found down in the Jiffy Server or hosted in external server. Below command should be run using 'root' login.
  Example:
  To start PostgreSQL server: systemctl start postgresql-9.6.service
• Once all this is done, the 'Internal Server Error' shouldbe fixed.
• Refresh or re-login to Jiffy to continue working.

502 Bad Gateway Error(for versions 2.7 and above)

Cause: This error occurs on the Jiffy front end when the downstream Gunicorn app server is down or unable to start. The 'Nginx' would be up and running fine which is the Web Server for Jiffy Application.

Solution:

• Login to Jiffy Server using putty or equivalent and switch to 'jiffyapp-usr'.
• Verify the Jiffy running processes in Jiffy Server using below command.
  • To checks and shows complete Jiffy processes running status: application status
  • Supervisor command: supervisorctl status
  
  
• Here, guincorn is down, which is Jiffy App server.
• Start the same using the following command:
  supervisorctl start gunicorn
• Now that the process has started successfully, refresh or re-login to Jiffy.

Unable to load Jiffy Web UI page

Cause: This error occurs when trying to load Jiffy URL in the browser and the wrong URL is being used or when the Jiffy Web Server 'nginx' is down.

Solution:

• Check if the correct URL is being used: http://<JiffyServerIP:8484> .
• If the URL is correct, then checking the 'nginx' process in Jiffy Server as a root user.
• Connect to the Jiffy Server using putty or equivalent with the 'root' login or switch to 'sudo' user. Check the status of the 'nginx' process using the following command:
  ps -ef | grep nginx
  
  
• If the process is not running, start the RabbitMQ using the following command:
  nginx OR
  /opt/nginx/sbin/nginx
• Check the status again using the following command:
  ps -ef | grep nginx
  
• Now the GUI should start loading.

JDI is down or not GUI not loading

Cause: This error occurs when JDI (Jiffy Data Interface) page fails to load at the URL https://<JiffyServerIP:8002>

Solution:

• Login to Jiffy Server using putty or equivalent as a 'jiffyapp-usr'.
• Check status of the JDI process using the following command:
  ps -ef | grep -i JDI OR
  supervisorctl status JDI
• If the process is not running, start the JDI using the following command:
  nginx OR
  /opt/nginx/sbin/nginx
• Check the status again using the following command:
  supervisorctl start JDI
• Now the JDI page should start loading.