CERN Accelerating science

REST

FTS

Sub-task

  • [FTS-830] - Publish error reason with FAILED state transitions
  • [FTS-831] - Prepend STAGING to the error message when caused on a staging operation
  • [FTS-832] - Publish start and end timestamps to state transitions as well

Epic

  • [FTS-828] - Provide extra information for staging performance analysis

Bug

  • [FTS-629] - Improve staging query
  • [FTS-718] - CLI should interpret nulls as nulls
  • [FTS-738] - fts-transfer-submit man page needs a refresh
  • [FTS-741] - Remove workaround for FTS-740
  • [FTS-773] - Consolidate default handling
  • [FTS-783] - Race condition with Producer inside MySqlAPI
  • [FTS-786] - If a transfer is to be marked as stalled, it better be killed
  • [FTS-791] - Tight loops when DB connection is lost
  • [FTS-797] - Cloud storage wildcard should be explicit (i.e '*')
  • [FTS-806] - Issue an SRM abort even if the bring online get status fails
  • [FTS-811] - Public share with 0 is automatically created if there is a link configuration
  • [FTS-816] - MySQL 5.7: ORDER BY clause is not in SELECT list
  • [FTS-835] - src_hostname and dst_hostname should not send the url schema
  • [FTS-840] - Deadlock between updateProtocol and updateFileTransferStatus
  • [FTS-841] - Incorrect timestamp for SUBMITTED state
  • [FTS-845] - Need a 404.html template
  • [FTS-849] - updateBringOnlineToken generates a bad query, and some files are left with an empty token
  • [FTS-853] - Draining ineffective

New Feature

  • [FTS-858] - Allow to disable cert verification

Task

  • [FTS-597] - Drop unused tables
  • [FTS-603] - Provide FTS via containers
  • [FTS-605] - Drop client side SOAP configuration
  • [FTS-780] - Provide backup duration in the logs
  • [FTS-792] - Drop snapshot API
  • [FTS-821] - Poll VOs to see which apparently unused functionalities are being used or will be
  • [FTS-824] - Drop t_server_sanity (unused)
  • [FTS-864] - Drop myosg-update
  • [FTS-865] - Reduce overview overhead

Improvement

  • [FTS-598] - Drop unused columns
  • [FTS-599] - Optimize database schema
  • [FTS-600] - Drop server side SOAP code
  • [FTS-617] - Redundant indexes
  • [FTS-621] - Use archive engine for t_*_backup tables
  • [FTS-671] - Restricted view for server logs in web monitoring
  • [FTS-727] - Remove workarounds for compatibility with schema 1.*
  • [FTS-770] - t_server_config values should be all picked per vo
  • [FTS-771] - Producer should connect using TLS
  • [FTS-772] - Remove dn from webmon
  • [FTS-779] - Do not print the user-dn on the url-copy logs
  • [FTS-782] - Heuristics for timeout is too short for small files
  • [FTS-807] - Make stalled timeout configurable, and set to 360 seconds
  • [FTS-808] - Log timeout events on Bring Online
  • [FTS-817] - Adapt to new database schema
  • [FTS-818] - 'credential' parameter removed from REST
  • [FTS-822] - Use ENUM for the states
  • [FTS-823] - Review queries
  • [FTS-827] - Replace EOT with a space on the messages
  • [FTS-847] - CLI should log warnings from CURL too
  • [FTS-859] - Remove wait timeout from storage blacklisting

Configuration Change

  • [FTS-843] - Change staging defaults to bulk of 500 and 400 requests

REST API

Bug

  • [FTS-546] - Cancel jobs when job_finished = NULL
  • [FTS-568] - FTS REST and VOMS RFC proxies
  • [FTS-734] - Fix schema reporting in REST API
  • [FTS-744] - Packaging: /var/log/fts3rest must be owned by fts3.fts3
  • [FTS-784] - Use always UTC for timestamp comparison in http
  • [FTS-798] - Client implementation of the delegation broken due to m2crypto API changes
  • [FTS-836] - fts-rest-http-authz-signed-cert error when comparing datetimes

New Feature

  • [FTS-705] - FTS can decide to enable or not session reuse depending on the size of the files contained on a job

Task

  • [FTS-408] - Consult roles for job listing
  • [FTS-603] - Provide FTS via containers
  • [FTS-792] - Drop snapshot API

Improvement

  • [FTS-727] - Remove workarounds for compatibility with schema 1.*
  • [FTS-781] - All server config now can be done per VO
  • [FTS-850] - Allow staging jobs for all protocols
  • [FTS-859] - Remove wait timeout from storage blacklisting

Bug

  • [FTS-752] - Rest sends the SUBMITTED/STAGING with the hostname rather than the alias
  • [FTS-719] - useexisting is deprecated, switch to extend_existing
  • [FTS-722] - Requires python-requests

Improvement

  • [FTS-749] - 40x statuses are not service errors, so they should be logged as info

New experimental feature: self-documenting API

Now the develop branch of the FTS3 RESTful API implements the auto-generation of documentation using the Swagger format, which is based on JSON.

The advantage of using this specific format is the possibility of using the Swagger UI, which allows to render more nicely the documentation.

You can see it in action in http://aalvarez.web.cern.ch/aalvarez/fts3api/

The REST API version 3.1.0 has been released into the stage repository, so it has already been applied to the FTS3 Pilot Service.

This release of the REST API start following the versioning schema of FTS3, meaning that REST 3.1.* must be compatible with FTS 3.1.* (if it is not, it is a bug!), but NOT with 3.2.* or 3.0.*

Bug

  • [FTS-11] - InvalidRequestError: Can't reconnect until invalid transaction is rolled back
  • [FTS-12] - REST allows any user to get the proxy request from any other
  • [FTS-18] - Not possible to get the transfers of a job with a NIL VO
  • [FTS-20] - hashed_id wrong value for reuse transfers
  • [FTS-21] - Proxy termination time flips between submissions

New Feature

  • [FTS-13] - Add basic file functionality
  • [FTS-14] - FTS3 Rest must accept activity parameter
  • [FTS-15] - Allow removal of delegated credentials
  • [FTS-17] - Get all jobs from a user, not only active ones

Task

  • [FTS-16] - Use readdirpp instead of readdir for listing the directory content