Commit 1ec8662e authored by Aleksandr Konstantinov's avatar Aleksandr Konstantinov

Merge branch 'master' into scitokens

parents dee6daa1 d3e81336
This diff is collapsed.
Advanced Resource Connector (ARC)
=================================
Release Notes for NorduGrid ARC 6.5.0 14.02.2020
------------------------------------------------------
We are happy to announce the ARC 6.5.0 release. This release comes with quite a large set of enhancements and improvements.
Bug 3841 (jobs piling up in FINISHING state) which one known site suffers with, is proving resilient.
We believe that a busy file system hosting the job's session directory had strong negative impact on the data staging.
This needs more investigation and future development to optimize the load on session directories.
However bug 3890 related to xrootd transfers, which had to be reverted in release 6.4.1, is now solved.
Highlights in this release
--------------------------
Of enhancements and new features we would like to highlight:
Arcctl:
* arcctl can now be installed standalone without A-REX. It comes with the test CA and 3rd party deploy (CA certs, VOMS) on board.
* As before you get the full arcctl if you install A-REX
Client area enhancements:
* installation automated with standalone arcctl
* new consistent and streamlined submission endpoint selection options
* consistent target-area oriented globus plugins split
* bash completion for client tools
* documentation for clients!
ARCHERY management:
* new JSON configuration for flexible services topology definition
Technology preview:
This release also includes the technology preview of a new functionality:
The "Community-defined RTEs" ecosystem that enables automated software environment provisioning on ARC CEs by community software managers.
The ecosystem consists of two layers: An ARCHERY-based software (RTE) registry and a new set of ARC control tool modules.
More details can be found in documentation: http://www.nordugrid.org/arc/arc6/admins/details/rtes_community.html
The following Bugzilla tickets have been resolved
---------------------------------------------------
* 3907 Implement the community-defined RTEs provisioning proposal
* 3905 More flexible service endpoints and grouping definition for archery-manage
* 3904 New consistent submission endpoints selection for ARC6 client
* 3903 Decouple arcctl from the A-REX package to be used independently
* 3895 Implement minwalltime and mincputime manual configuration in arc.conf
* 3891 A-REX crashes stemming from Logger
* 3890 xrootd transfers don't work
* 3887 MaxWallTime and MaxCPUTime not propagated to infosys
* 3885 A-rex crashed (Duplicate of 3890)
* 3880 logs created as root user when user= defined in [arex] block causes the service not to start
* 3838 Ensure space trimming in block names all over the code
* 3538 ARC Client libraries should take the batch system queue name from GLUE2 ComputingShare.MappingQueue and NOT from ComputingShare.Name
Documentation
==============
The ARC 6 documentation has been rewritten in a modern format and is
placed at http://www.nordugrid.org/arc/arc6
If you miss something or have questions, please contact us!
Installing ARC 6
=================
We recommend to install ARC release using the Nordugrid repository.
* Start with the basics: http://www.nordugrid.org/arc/arc6/admins/try_arc6.html
* Get production ready: http://www.nordugrid.org/arc/arc6/admins/arc6_install_guide.html
Note that if you instead install from EPEL for RHEL 7 compatible systems,
the ARC 6 packages can be found as nordugrid-arc6-* and not nordugrid-arc-*
This was needed in order to supply both ARC 5 already available for RHEL7 and the new ARC 6.
We provide binary builds for all supported versions of
* Debian (i386 and amd64)
* Fedora (i386 and x86_64)
* CentOS EL6 (i386 and x86_64) and EL7/EL8 (x86_64)
* Ubuntu (i386 and amd64, both LTS and interim releases)
In addition we build nightly packages for older platforms.
Future support of ARC 5-series
===============================
Note that now that ARC 6 is released, we will only provide security updates of ARC 5.
In particular:
1) No new feature development is planned or going on for ARC5 and no bug-fixing development
will happen on ARC5 code base in the future except for security issues.
2) Security fixes for ARC5 will be provided till end of June 2020.
3) Production Sites already running ARC 5 will be able to get deployment and configuration
troubleshooting help via GGUS till end June 2021. This we call "operational site support".
4) ARC5 is available in EPEL7 and will stay there. EPEL8 will only contain ARC 6.
Getting in touch
==================
If you need help with the configuration file or have any other questions related to ARC 6, please contact us via
* Email: nordugrid-discuss@nordugrid.org
* Skype channel dedicated to ARC 6 support: https://join.skype.com/dyf3A6Uutjy2
......@@ -357,10 +357,10 @@ using namespace Arc;
}
DataStatus DataPointHTTP::do_stat_http(URL& rurl, FileInfo& file) {
PayloadRaw request;
PayloadRawInterface *inbuf = NULL;
HTTPClientInfo info;
for(int redirects_max = 10;redirects_max>=0;--redirects_max) {
PayloadRawInterface *inbuf = NULL;
HTTPClientInfo info;
PayloadRaw request;
std::string path = rurl.FullPathURIEncoded();
info.lastModified = (time_t)(-1);
AutoPointer<ClientHTTP> client(acquire_client(rurl));
......@@ -485,13 +485,13 @@ using namespace Arc;
}
std::multimap<std::string, std::string> propattr;
propattr.insert(std::pair<std::string, std::string>("Depth","0"));
PayloadRawInterface *inbuf = NULL;
HTTPClientInfo info;
for(int redirects_max = 10;redirects_max>=0;--redirects_max) {
std::string path = rurl.FullPathURIEncoded();
info.lastModified = (time_t)(-1);
AutoPointer<ClientHTTP> client(acquire_client(rurl));
if (!client) return DataStatus::StatError;
PayloadRawInterface *inbuf = NULL;
HTTPClientInfo info;
info.lastModified = (time_t)(-1);
MCC_Status r = client->process("PROPFIND", path, propattr, &request, &info, &inbuf);
if (!r) {
if (inbuf) { delete inbuf; inbuf = NULL; }
......@@ -566,13 +566,13 @@ using namespace Arc;
}
std::multimap<std::string, std::string> propattr;
propattr.insert(std::pair<std::string, std::string>("Depth","1")); // for listing
PayloadRawInterface *inbuf = NULL;
HTTPClientInfo info;
for(int redirects_max = 10;redirects_max>=0;--redirects_max) {
std::string path = rurl.FullPathURIEncoded();
info.lastModified = (time_t)(-1);
AutoPointer<ClientHTTP> client(acquire_client(rurl));
if (!client) return DataStatus::StatError;
PayloadRawInterface *inbuf = NULL;
HTTPClientInfo info;
info.lastModified = (time_t)(-1);
MCC_Status r = client->process("PROPFIND", path, propattr, &request, &info, &inbuf);
if (!r) {
if (inbuf) { delete inbuf; inbuf = NULL; }
......@@ -962,10 +962,10 @@ using namespace Arc;
std::string path = point.CurrentLocation().FullPathURIEncoded();
DataStatus failure_code;
if (!client) return false;
HTTPClientInfo transfer_info;
PayloadRaw request;
PayloadStreamInterface *instream = NULL;
for(;;) { // for retries
HTTPClientInfo transfer_info;
PayloadRaw request;
PayloadStreamInterface *instream = NULL;
MCC_Status r = client->process(ClientHTTPAttributes("GET", path),
&request, &transfer_info,
&instream);
......@@ -1248,8 +1248,6 @@ using namespace Arc;
URL client_url = point.url;
AutoPointer<ClientHTTP> client(point.acquire_client(client_url));
if (!client) return false;
HTTPClientInfo transfer_info;
PayloadRawInterface *response = NULL;
std::string path = client_url.FullPathURIEncoded();
// TODO: Do ping to *client in order to check if connection is alive.
// TODO: But ping itself can destroy connection on 1.0-like servers.
......@@ -1269,6 +1267,8 @@ using namespace Arc;
}
StreamBuffer request(*point.buffer);
if (point.CheckSize()) request.Size(point.GetSize());
PayloadRawInterface *response = NULL;
HTTPClientInfo transfer_info;
MCC_Status r = client->process(ClientHTTPAttributes("PUT", path, attrs),
&request, &transfer_info, &response);
if (response) { delete response; response = NULL; }
......@@ -1364,6 +1364,7 @@ using namespace Arc;
MCC_Status r = client->process("PUT", path, &request, &transfer_info,
&response);
if (response) delete response;
response = NULL;
if (!r) {
client = NULL;
// Failed to transfer chunk - retry.
......@@ -1432,6 +1433,7 @@ using namespace Arc;
MCC_Status r = client->process("PUT", path, &request, &transfer_info,
&response);
if (response) delete response;
response = NULL;
if (!r) {
client = NULL;
if ((++retries) > 10) {
......
......@@ -246,7 +246,7 @@ namespace Arc {
if (repmsg.Payload() != NULL) {
try {
*response = dynamic_cast<PayloadStreamInterface*>(repmsg.Payload());
repmsg.Payload(NULL);
if(*response) repmsg.Payload(NULL);
} catch (std::exception&) {
}
delete repmsg.Payload();
......@@ -281,7 +281,7 @@ namespace Arc {
if (repmsg.Payload() != NULL) {
try {
*response = dynamic_cast<PayloadStreamInterface*>(repmsg.Payload());
repmsg.Payload(NULL);
if(*response) repmsg.Payload(NULL);
} catch (std::exception&) {
}
delete repmsg.Payload();
......@@ -468,7 +468,7 @@ namespace Arc {
if (mresp != NULL) {
try {
*response = dynamic_cast<PayloadRawInterface*>(mresp);
mresp = NULL;
if(*response) mresp = NULL;
} catch (std::exception&) {
}
delete mresp;
......@@ -577,7 +577,7 @@ namespace Arc {
if (mresp != NULL) {
try {
*response = dynamic_cast<PayloadRawInterface*>(mresp);
mresp = NULL;
if(*response) mresp = NULL;
} catch (std::exception&) {
}
delete mresp;
......@@ -595,7 +595,7 @@ namespace Arc {
if (mresp != NULL) {
try {
*response = dynamic_cast<PayloadRawInterface*>(mresp);
mresp = NULL;
if(*response) mresp = NULL;
} catch (std::exception&) {
}
delete mresp;
......@@ -613,7 +613,7 @@ namespace Arc {
if (mresp != NULL) {
try {
*response = dynamic_cast<PayloadStreamInterface*>(mresp);
mresp = NULL;
if(*response) mresp = NULL;
} catch (std::exception&) {
}
delete mresp;
......@@ -631,7 +631,7 @@ namespace Arc {
if (mresp != NULL) {
try {
*response = dynamic_cast<PayloadStreamInterface*>(mresp);
mresp = NULL;
if(*response) mresp = NULL;
} catch (std::exception&) {
}
delete mresp;
......@@ -735,7 +735,7 @@ namespace Arc {
if (repmsg.Payload() != NULL) {
try {
*response = dynamic_cast<PayloadSOAP*>(repmsg.Payload());
repmsg.Payload(NULL);
if(*response) repmsg.Payload(NULL);
} catch (std::exception&) {
}
delete repmsg.Payload();
......@@ -772,7 +772,7 @@ namespace Arc {
if (repmsg.Payload() != NULL) {
try {
*response = dynamic_cast<PayloadSOAP*>(repmsg.Payload());
repmsg.Payload(NULL);
if(*response) repmsg.Payload(NULL);
} catch (std::exception&) {
}
delete repmsg.Payload();
......
......@@ -29,7 +29,7 @@ sub prioritizedvalues {
}
# just in case all the above fails, return empty string
$log->debug("No suitable value found in call to prioritizedvalues. Returning empty string");
$log->debug("No suitable value found in call to prioritizedvalues. Returning undefined");
return undef;
}
......
......@@ -666,8 +666,8 @@ sub prioritizedvalues {
}
# just in case all the above fails, log and return empty string
$log->debug("No suitable value found in call to prioritizedvalues. Returning empty string");
return '';
$log->debug("No suitable value found in call to prioritizedvalues. Returning undefined");
return undef;
}
# TODO: add VOs information
......
......@@ -293,7 +293,7 @@ class RTEControl(ComponentControl):
self.__list_rte(self.system_rtes, args.long)
elif args.user:
self.__list_rte(self.user_rtes, args.long)
elif args.community:
elif hasattr(args, 'community') and args.community:
self.__list_rte(self.community_rtes, args.long)
elif args.available:
self.system_rtes.update(self.user_rtes)
......
......@@ -66,7 +66,7 @@ class TestCAControl(ComponentControl):
try:
# try to get it from hostname -f
hostname_f = subprocess.Popen(['hostname', '-f'], stdout=subprocess.PIPE)
self.hostname = hostname_f.stdout.readline().strip()
self.hostname = hostname_f.stdout.readline().decode().strip()
self.logger.debug('Using hostname from \'hostname -f\': %s', self.hostname)
except OSError:
# fallback
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment