FNE / FNX troubleshooting
- Where are the log files?
- How to update FNX/FNE license servers' software?
- FNE registration problems
- FNE log file: errors and messages
- What is /var/opt/flexnetls/socbu/flexnetls_licenses.mv.db and can it be deleted?
- What is directory /var/opt/flexnetls/socbu/logs/ used for and can it be deleted?
Where are the log files?
You can find the log files in the following folders for
- FNE server:
/var/opt/flexnetls/socbu/logs > filename flexnetls*.log*andflexnetls.critical.log - FNE server viewer:
/var/opt/flexnetls/viewer/logs > filename fne-viewer.log* - Proxy:
/opt/flexlm/log > filename proxy_socbu*.log
How to update FNX/FNE license servers' software?
See also - Failover license server installation, upgrades and updates.
It is the advantage of the FNE/FNX server, that an update works without downtime. Do the following steps:
Note: If there is not a failover or tertiary server existing, you can skip the steps accordingly.
- Test the integrity of the downloaded SW on the primary license server:
sh install-fne-<date>-<version>.sh test
If you get the message "Package integrity verified" you can proceed with the other steps. - Get emergency license for standby emergency license server and install it - just to be prepared
- Primary server: Do a dry-run installation on the primary server and look for messages. Example for the dry-run command:
sudo sh install-fne-<date>-<version>.sh dry-run - Primary server: If there are not any problems reported with the dry-run, do the update on the primary server. Example for the update command (no option used):
sudo sh install-fne-<date>-<version>.sh - Check e.g. with
fne-status, that the primary server is running. - Wait 2 minutes.
- Failover server: Do the update also on the failover server. See command above.
- Check e.g. with
fne-status, that the failover server is running. - Wait 2 minutes.
- Check with
fne-status, if the FNE failover and (if installed) FNP triad is up again. - Tertiary server: Do the update also on the tertiary server.
- Check e.g. with
fne-status, that all servers are running.
If there are any problems, please check, if a switch to the standby emergency license server is reasonable and open a LIST ticket to report the problem.
FNE registration problems
Error "Connection to https://license.advantest.com/flexnet/deviceservices failed with exception"
Issue description:
The user tried command
register-fne-server <id>
with the appropriate <id> and got the following error message:
Failed to activate your server
\{
"key" : "glsErr.connectionFailed",
"message": "Connection to https://license.advantest.com/flexnet/deviceservices failed with exception: exception: error on POST request for \ \"https://license.advantest.com/fL
exnet/deviceservices\": Connection reset; nested exception is javax.net.ssl.SSLException: Connection reset,*
: [ "https://license.advantest.com/flexnet/deviceservices, I/0 error on POST request for \"https://license.advantest.com/flexnet/deviceservices\": Connec
tion reset; nested exception is javax.net.ssl.SSLException: Connection reset ]*
Background information:
The connection to license.advantest.com with port 443 was not really available, but for the FNE server software it looked like that a connection is possible, therefor it tried and failed.
Solution: Enter the line
0.0.0.0 license.advantest.com
to file /etc/hosts.
FNE log file: errors and messages
Installation problems - error "/tmp/: failed to map segment from shared object: Operation not permitted"
Creation date: 08-Jan-2020
Author: Heike Klein
Issue description: The FNE server has been installed but cannot be started.
Log file error in /var/opt/flexnetls/socbu/logs/flexnetls.log.<date> - highlighted in red:
00:02:49,408 advantest-fne-server start
00:02:55,431 Used Options: /usr/lib/jvm/java-1.8.0-ibm-1.8.0.5.40-1jpp.1.el7.x86_64/jre/bin/java -Dbase.dir=/var/opt/flexnetls/socbu -Djsse.enableSNIExtension=false -server -Xms2g -Xmx2g -XX:CompressedClassSpaceSize=64m -XX:MetaspaceSize=256m -XX:+UseG1GC -XX:NewRatio=3 -XX:MaxGCPauseMillis=75 -XX:G1HeapWastePercent=10 -XX:InitiatingHeapOccupancyPercent=75 -XX:+CMSScavengeBeforeRemark -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED -XX:+ScavengeBeforeFullGC -Djava.security.egd=file:/dev/./urandom -jar /opt/flexnetls/socbu/flexnetls.jar -server-alias thbnk01fnxspl1v.th-bnk01.nxp.com -producer-settings /opt/flexnetls/socbu/producer-settings.xml -port 7070 -https-client-configuration /var/opt/flexnetls/socbu/outgoing-certs.conf
00:02:55,438 Environment: Version 2019.07.1.0-19315 / 7.6GB mem / 3.1GB disk free / 4x Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz / Red Hat Enterprise Linux Server release 7.6 (Maipo)
00:02:58,835 INFO main Starting FlexNet License Server 2019.07 (build 256892)
00:02:58,836 INFO main Copyright (C) 2013-2019 Flexera Software LLC.
00:02:58,836 INFO main All Rights Reserved.
00:02:58,836 INFO main Running as a service
00:03:02,575 ERROR main Native code loading error java.lang.UnsatisfiedLinkError: /tmp/llsnc-2019.07.1.0-256892-amd64-libconnector.so(/tmp/llsnc-2019.07.1.0-256892-amd64-libconnector.so: failed to map segment from shared object: Operation not permitted)
at java.lang.ClassLoader.loadLibraryWithPath(ClassLoader.java:1453)
at java.lang.System.load(System.java:570)
Reason for error:
the servers tries to use/exceute a shared library located in the /tmp directory. This is not allowed on the appropriate system. Running executables from /tmp is forbidden by the noexec option in the mount flags: check the /etc/fstab entry which looks like
/dev/mapper/vg00-tmp /tmp ext4 defaults,nodev,nosuid,noexec 1 2
Solution:
Ask IT to remove the noexec option to /tmp in /etc/fstab and remount it withmount -o remount,exec /tmp
on all affected FNE license servers.
server filter checkout rejection
In the FNE log file flexnetls.log you maybe find the following message:server filter checkout rejection
This message is basically saying the license requested is not available, f.e. for a specific serial number.
This is not an error of the server, its just the way the server returns a request for a license that is not present.
TS storage corrupted - FNE server cannot be started - FNE log error "FATAL main IO Exception: null"
In the FNE log file flexnetls.log you find the following entries:
03:36:16,826 INFO main Starting FlexNet License Server 2019.07 (build 256892)
03:36:16,827 INFO main Copyright (C) 2013-2019 Flexera Software LLC.
03:36:16,827 INFO main All Rights Reserved.
03:36:16,827 INFO main Running as a service
03:36:21,736 FATAL main IO Exception: null [90028-196]
Note: you can extract the log file errors with command: showlog-fne-server -2
The error indicates a corrupted TS storage. In order to repair it, do the following:
cd /opt/flexnetls/socbusudo ./reset-fne-server -force- If there is a failover license server configured you need to run the following command (not as root!) on the primary license server
./fne-failover-setup repair primary <failover_host.domain>
This will repair the primary license server without touching the secondary license server.Note: the repair option is available since June 2020. It is not available if an older FNE/FNX installation script has been used and the SW has not been updated since then. Please check with fne-failover-setup -help if the option is available.
- If the failover server is corrupt, then you need to run the following command (not as root!) on the primary license server (this is not a typo - you need to run it on the primary license server!):
fne-failover-setup repair failover <failover_host.domain>Note: The command
fne-failover-setup uses ssh/scpandsudo. You may be asked for your password several times.
Note: you can check the help page of fne-failover-setup with: fne-failover-setup -help
What is /var/opt/flexnetls/socbu/flexnetls_licenses.mv.db and can it be deleted?
/var/opt/flexnetls/socbu/flexnetls_licenses.mv.db is the database of the FNE server and should not be deleted at all.
What is directory /var/opt/flexnetls/socbu/logs/ used for and can it be deleted?
/var/opt/flexnetls/socbu/logs/ contain the log files of the FNE server and is needed for analysis if something is going wrong. The logs directory is always around 12GB big: it is cleaned up to 10GB each day and on head is the up-to-date log file.