diff --git a/docs/develop/apps/haikudepot/images/process-interplay.svg b/docs/develop/apps/haikudepot/images/process-interplay.svg new file mode 100644 index 0000000000..835e33779b --- /dev/null +++ b/docs/develop/apps/haikudepot/images/process-interplay.svg @@ -0,0 +1,612 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Process A + + + + + + + + + + + + + + + Process B + + + + + + + + Process C + + + + + + + + ProcessNode A + + + + + + + + ProcessNode B + + + + + + + + ProcessNode C + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Model + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Coordinator + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + MainWindow + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Local DiskSystem + + + + + + + + \ No newline at end of file diff --git a/docs/develop/apps/haikudepot/images/processes.svg b/docs/develop/apps/haikudepot/images/processes.svg new file mode 100644 index 0000000000..d5f3c632ad --- /dev/null +++ b/docs/develop/apps/haikudepot/images/processes.svg @@ -0,0 +1,173 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + LocalRepositoryUpdateProcess + + + + + + + + LocalPkgDataLoadProcess + + + + + + + + ServerRepositoryDataUpdateProcess + + + + + + + + ServerIconExportUpdateProcess + + + + + + + + ServerPkgDataUpdateProcess + + + + + + + + ServerPkgDataUpdateProcess + + + + + + + + ServerPkgDataUpdateProcess + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/develop/apps/haikudepot/server.md b/docs/develop/apps/haikudepot/server.md new file mode 100644 index 0000000000..4a9b35c4c3 --- /dev/null +++ b/docs/develop/apps/haikudepot/server.md @@ -0,0 +1,60 @@ +# HaikuDepot and Server Interactions + +## Introduction + +This document aims to outline the general approach taken within the HaikuDepot application with regard to coordinating processes that relate to fetching and consuming data from remote systems. + +There are two main sources of remote data that are downloaded and consumed from network sources into the HaikuDepot desktop application; + +* Repository HPKR data from a Haiku mirror such as "HaikuPorts" +* Meta-data related to packages from [HaikuDepotServer](http://depot.haiku-os.org) (HDS) such as icons, localizations, ratings and so on. + +## Process, ProcessNode and Coordinator + +A _Process_ (root class ```AbstractProcess```) is a class that takes responsibility for some aspect of pulling material down from a network source and processing it. + +A _ProcessNode_ is a holder for a Process, but also takes responsibility for the following; + +* Maintaining the relationship between the Processes. For example, if Process A needs to complete before Process B then the ProcessNode would record this fact. It does this by storing _predecessor_ and _successor_ ProcessNodes. +* Starting the held Process in a newly spawned thread. +* Stopping the held Process. + +A _Coordinator_ holds a list of ProcessNodes. It will start, stop and cancel nodes as necessary such that, in an ideal case, the various ProcessNodes are completed in the correct order. + +The _ProcessCoordinatorFactory_ is able to create Coordinators. + +## Bulk Load Processes + +The following diagram shows the logical dependencies of the various Processes that are involved in refreshing the HPKR data from remote repositories and then loading data from the HDS system. + +![Process Dependencies](images/processes.svg) + +For example, the ```ServerRepositoryDataUpdateProcess``` must wait until the ```LocalRepositoryUpdateProcess``` has completed before it is able to be started. It is the reponsibility of the Coordinator to ensure that this sequencing is enforced. There are many instances of ```ServerPkgDataUpdateProcess``` shown because there will be one launched for each of the Repositories for which data will be downloaded; "HaikuDepot" etc... + +## Process / ProcessNode / Coordinator + +The following diagram shows the relationship and interplay between the various objects that are involved in running a larger task. Only fictional Processes are shown to keep the diagram tidy. See above for the actual Processes. + +![Process Relationship and Interplay](images/process-interplay.svg) + +Dotted lines show associations between elements and red lines show interaction or data-flow. Green arrows here demonstratively show some dependency; Process C cannot start until A and B are completed. + +The MainWindow owns the Coordinator for the life-span of undertaking some larger task. + +Each Process is coupled with a ProcessNode and then the Coordinator has a list of the ProcessNodes-s. The Processes are generally writing to the local disk system (often with compressed files) to cache data (see ```~/config/cache/HaikuDepot```) and also relay data into the ```Model``` object that maintains state for the HaikuDepot desktop application. + +The Processes communicate when they have finished to the Coordinator and it is at these events that the Coordinator is able to introspect the state of the Processes in order to know what to do next. + +The Coordinator also communicates with MainWindow. It communicates with the MainWindow in order to signal changes or progress in the overall larger task. The MainWindow also uses these events to discover when the Coordinator has completely finished. + +## Failure + +A Process may fail or be stopped. If a Process fails or is stopped then successor Processes, or those that would have run after the failed process, are stopped so that they will not run. + +The Coordinator will still try to complete any other Processes that could still run or are running already. + +Upon the Coordinator completing, the Coordinator will signal to the MainWindow client the change in state and then the MainWindow will be able to identify that the Coordinator has completed, but that something has gone wrong along the way. + +## Concurrency + +It is important to note that Processes may run concurrently. The Processes' are modelled by the Coordinator as a list rather than a tree. The dependencies are likely to form a tree or web of Processes that dictates the order of execution, but it is also quite possible to have multiple non-intersecting trees or webs such that Processes will execute independently. diff --git a/src/apps/haikudepot/HaikuDepotConstants.h b/src/apps/haikudepot/HaikuDepotConstants.h index edb5520b05..d251f69c20 100644 --- a/src/apps/haikudepot/HaikuDepotConstants.h +++ b/src/apps/haikudepot/HaikuDepotConstants.h @@ -16,13 +16,14 @@ enum { MSG_NETWORK_TRANSPORT_ERROR = 'nett', MSG_SERVER_ERROR = 'svre', MSG_SERVER_DATA_CHANGED = 'svdc', + MSG_ALERT_SIMPLE_ERROR = 'nser', MSG_DID_ADD_USER_RATING = 'adur', MSG_DID_UPDATE_USER_RATING = 'upur' }; -#define RATING_MISSING -1.0f -#define RATING_MIN 0.0f +#define RATING_MISSING -1.0f +#define RATING_MIN 0.0f #define HD_ERROR_BASE (B_ERRORS_END + 1) @@ -32,8 +33,16 @@ enum { #define HD_ERR_NO_DATA (HD_ERROR_BASE + 4) -#define REPOSITORY_NAME_SYSTEM "system" -#define REPOSITORY_NAME_INSTALLED "installed" +#define REPOSITORY_NAME_SYSTEM "system" +#define REPOSITORY_NAME_INSTALLED "installed" + + +#define KEY_ALERT_TEXT "alert_text" +#define KEY_ALERT_TITLE "alert_title" +#define KEY_WORK_STATUS_TEXT "work_status_text" +#define KEY_WORK_STATUS_PROGRESS "work_status_progress" +#define KEY_WINDOW_SETTINGS "window_settings" +#define KEY_MAIN_SETTINGS "main_settings" // These constants reference resources in 'HaikuDepot.ref' diff --git a/src/apps/haikudepot/Jamfile b/src/apps/haikudepot/Jamfile index b659510a9e..dc76250103 100644 --- a/src/apps/haikudepot/Jamfile +++ b/src/apps/haikudepot/Jamfile @@ -69,8 +69,6 @@ Application HaikuDepot : MarkupTextView.cpp MessagePackageListener.cpp Model.cpp - BulkLoadContext.cpp - BulkLoadStateMachine.cpp PackageAction.cpp PackageActionHandler.cpp PackageContentsView.cpp @@ -98,23 +96,30 @@ Application HaikuDepot : DumpExportRepositorySource.cpp DumpExportRepositoryJsonListener.cpp - # network + server + # network + server / local processes + AbstractProcess.cpp AbstractServerProcess.cpp AbstractSingleFileServerProcess.cpp + LocalPkgDataLoadProcess.cpp + LocalRepositoryUpdateProcess.cpp + ProcessCoordinator.cpp + ProcessCoordinatorFactory.cpp + ProcessNode.cpp ServerHelper.cpp ServerSettings.cpp - WebAppInterface.cpp - PkgDataUpdateProcess.cpp - RepositoryDataUpdateProcess.cpp + ServerPkgDataUpdateProcess.cpp + ServerRepositoryDataUpdateProcess.cpp ServerIconExportUpdateProcess.cpp StandardMetaDataJsonEventListener.cpp StandardMetaData.cpp + WebAppInterface.cpp # tar TarArchiveHeader.cpp TarArchiveService.cpp #util + AppUtils.cpp DataIOUtils.cpp RepositoryUrlUtils.cpp StorageUtils.cpp diff --git a/src/apps/haikudepot/List.h b/src/apps/haikudepot/List.h index 186801875f..c32651e5ab 100644 --- a/src/apps/haikudepot/List.h +++ b/src/apps/haikudepot/List.h @@ -1,5 +1,6 @@ /* * Copyright 2009-2013, Stephan Aßmus + * Copyright 2018, Andrew Lindesay * All rights reserved. Distributed under the terms of the MIT License. */ #ifndef LIST_H @@ -20,12 +21,28 @@ template class List { typedef List SelfType; + typedef int32 (*CompareItemFn)(const ItemType& one, const ItemType& two); + typedef int32 (*CompareContextFn)(const void* context, + const ItemType& item); public: List() : fItems(NULL), fCount(0), - fAllocatedCount(0) + fAllocatedCount(0), + fCompareItemsFunction(NULL), + fCompareContextFunction(NULL) + { + } + + List(CompareItemFn compareItemsFunction, + CompareContextFn compareContextFunction) + : + fItems(NULL), + fCount(0), + fAllocatedCount(0), + fCompareItemsFunction(compareItemsFunction), + fCompareContextFunction(compareContextFunction) { } @@ -33,9 +50,11 @@ public: : fItems(NULL), fCount(0), - fAllocatedCount(0) + fAllocatedCount(0), + fCompareItemsFunction(other.fCompareItemsFunction), + fCompareContextFunction(other.fCompareContextFunction) { - *this = other; + _AddAllVerbatim(other); } virtual ~List() @@ -49,29 +68,12 @@ public: SelfType& operator=(const SelfType& other) { - if (this == &other) - return *this; - - if (PlainOldData) { - if (_Resize(other.fCount)) - memcpy(fItems, other.fItems, fCount * sizeof(ItemType)); - } else { - // Make sure to call destructors of old objects. - // NOTE: Another option would be to use - // ItemType::operator=(const ItemType& other), but then - // we would need to be carefull which objects are already - // initialized. Also ItemType would be required to implement the - // operator, while doing it this way requires only a copy - // constructor. - _Resize(0); - for (uint32 i = 0; i < other.fCount; i++) { - if (!Add(other.ItemAtFast(i))) - break; - } - } + if (this != &other) + _AddAllVerbatim(other); return *this; } + bool operator==(const SelfType& other) const { if (this == &other) @@ -114,64 +116,40 @@ public: return fCount; } -/*! Note that the use of this method will depend on the list being ordered. +/*! Note that the use of this method will depend on the list being sorted. */ - inline int32 BinarySearch(const void* context, - int32 (*compareFunc)(const void* context, const ItemType& item)) + inline int32 Search(const void* context) const { - if (fCount == 0) + if (fCount == 0 || fCompareContextFunction == NULL) return -1; - return _BinarySearchBounded(context, compareFunc, 0, fCount - 1); + return _BinarySearchBounded(context, 0, fCount - 1); } - inline bool AddOrdered(const ItemType& copyFrom, - int32 (*compareFunc)(const ItemType& one, const ItemType& two)) - { - // special case - if (fCount == 0 - || compareFunc(copyFrom, ItemAtFast(fCount - 1)) > 0) { - return Add(copyFrom); - } - - return _AddOrderedBounded(copyFrom, compareFunc, 0, fCount - 1); - } + /*! This function will add the item into the list. If the list is sorted + then the item will be insert in order. If the list is not sorted then + the item will be inserted at the end of the list. + */ inline bool Add(const ItemType& copyFrom) { - if (_Resize(fCount + 1)) { - ItemType* item = fItems + fCount - 1; - // Initialize the new object from the original. - if (!PlainOldData) - new (item) ItemType(copyFrom); - else - *item = copyFrom; - return true; + if (fCompareItemsFunction != NULL) { + return _AddOrdered(copyFrom); } - return false; + + return _AddTail(copyFrom); } inline bool Add(const ItemType& copyFrom, int32 index) { - if (index < 0 || index > (int32)fCount) - return false; + // if the list is sorted then ignore the index and just insert in + // order. + if (fCompareItemsFunction != NULL) { + return _AddOrdered(copyFrom); + } - if (!_Resize(fCount + 1)) - return false; - - int32 nextIndex = index + 1; - if ((int32)fCount > nextIndex) - memmove(fItems + nextIndex, fItems + index, - (fCount - nextIndex) * sizeof(ItemType)); - - ItemType* item = fItems + index; - if (!PlainOldData) - new (item) ItemType(copyFrom); - else - *item = copyFrom; - - return true; + return _AddAtIndex(copyFrom, index); } inline bool Remove() @@ -210,6 +188,12 @@ public: inline bool Replace(int32 index, const ItemType& copyFrom) { + if (fCompareItemsFunction != NULL) { + bool result = Remove(index); + _AddOrdered(copyFrom); + return result; + } + if (index < 0 || index >= (int32)fCount) return false; @@ -259,11 +243,10 @@ public: private: inline int32 _BinarySearchLinearBounded( const void* context, - int32 (*compareFunc)(const void* context, const ItemType& item), - int32 start, int32 end) + int32 start, int32 end) const { for(int32 i = start; i <= end; i++) { - if (compareFunc(context, ItemAtFast(i)) == 0) + if (fCompareContextFunction(context, ItemAtFast(i)) == 0) return i; } @@ -271,35 +254,53 @@ private: } inline int32 _BinarySearchBounded( - const void* context, - int32 (*compareFunc)(const void* context, const ItemType& item), - int32 start, int32 end) + const void* context, int32 start, int32 end) const { if (end - start < BINARY_SEARCH_LINEAR_THRESHOLD) - return _BinarySearchLinearBounded(context, compareFunc, start, end); + return _BinarySearchLinearBounded(context, start, end); int32 mid = start + ((end - start) >> 1); - if (compareFunc(context, ItemAtFast(mid)) >= 0) - return _BinarySearchBounded(context, compareFunc, mid, end); - return _BinarySearchBounded(context, compareFunc, start, mid - 1); + if (fCompareContextFunction(context, ItemAtFast(mid)) >= 0) + return _BinarySearchBounded(context, mid, end); + return _BinarySearchBounded(context, start, mid - 1); } + inline void _AddAllVerbatim(const SelfType& other) + { + if (PlainOldData) { + if (_Resize(other.fCount)) + memcpy(fItems, other.fItems, fCount * sizeof(ItemType)); + } else { + // Make sure to call destructors of old objects. + // NOTE: Another option would be to use + // ItemType::operator=(const ItemType& other), but then + // we would need to be careful which objects are already + // initialized. Also ItemType would be required to implement the + // operator, while doing it this way requires only a copy + // constructor. + _Resize(0); + for (uint32 i = 0; i < other.fCount; i++) { + if (!Add(other.ItemAtFast(i))) + break; + } + } + } + + inline bool _AddOrderedLinearBounded( - const ItemType& copyFrom, - int32 (*compareFunc)(const ItemType& one, const ItemType& two), - int32 start, int32 end) + const ItemType& copyFrom, int32 start, int32 end) { for(int32 i = start; i <= (end + 1); i++) { bool greaterBefore = (i == start) - || (compareFunc(copyFrom, ItemAtFast(i - 1)) > 0); + || (fCompareItemsFunction(copyFrom, ItemAtFast(i - 1)) > 0); if (greaterBefore) { bool lessAfter = (i == end + 1) - || (compareFunc(copyFrom, ItemAtFast(i)) <= 0); + || (fCompareItemsFunction(copyFrom, ItemAtFast(i)) <= 0); if (lessAfter) - return Add(copyFrom, i); + return _AddAtIndex(copyFrom, i); } } @@ -308,18 +309,65 @@ private: } inline bool _AddOrderedBounded( - const ItemType& copyFrom, - int32 (*compareFunc)(const ItemType& one, const ItemType& two), - int32 start, int32 end) + const ItemType& copyFrom, int32 start, int32 end) { if(end - start < BINARY_SEARCH_LINEAR_THRESHOLD) - return _AddOrderedLinearBounded(copyFrom, compareFunc, start, end); + return _AddOrderedLinearBounded(copyFrom, start, end); int32 mid = start + ((end - start) >> 1); - if (compareFunc(copyFrom, ItemAtFast(mid)) >= 0) - return _AddOrderedBounded(copyFrom, compareFunc, mid, end); - return _AddOrderedBounded(copyFrom, compareFunc, start, mid - 1); + if (fCompareItemsFunction(copyFrom, ItemAtFast(mid)) >= 0) + return _AddOrderedBounded(copyFrom, mid, end); + return _AddOrderedBounded(copyFrom, start, mid - 1); + } + + inline bool _AddTail(const ItemType& copyFrom) + { + if (_Resize(fCount + 1)) { + ItemType* item = fItems + fCount - 1; + // Initialize the new object from the original. + if (!PlainOldData) + new (item) ItemType(copyFrom); + else + *item = copyFrom; + return true; + } + return false; + } + + + inline bool _AddAtIndex(const ItemType& copyFrom, int32 index) + { + if (index < 0 || index > (int32)fCount) + return false; + + if (!_Resize(fCount + 1)) + return false; + + int32 nextIndex = index + 1; + if ((int32)fCount > nextIndex) + memmove(fItems + nextIndex, fItems + index, + (fCount - nextIndex) * sizeof(ItemType)); + + ItemType* item = fItems + index; + if (!PlainOldData) + new (item) ItemType(copyFrom); + else + *item = copyFrom; + + return true; + } + + + inline bool _AddOrdered(const ItemType& copyFrom) + { + // special case + if (fCount == 0 + || fCompareItemsFunction(copyFrom, ItemAtFast(fCount - 1)) > 0) { + return _AddTail(copyFrom); + } + + return _AddOrderedBounded(copyFrom, 0, fCount - 1); } inline bool _Resize(uint32 count) @@ -348,10 +396,12 @@ private: return true; } - ItemType* fItems; - ItemType fNullItem; - uint32 fCount; - uint32 fAllocatedCount; + ItemType* fItems; + ItemType fNullItem; + uint32 fCount; + uint32 fAllocatedCount; + CompareItemFn fCompareItemsFunction; + CompareContextFn fCompareContextFunction; }; diff --git a/src/apps/haikudepot/model/Model.cpp b/src/apps/haikudepot/model/Model.cpp index e477bbe7ff..6b77a2bfe6 100644 --- a/src/apps/haikudepot/model/Model.cpp +++ b/src/apps/haikudepot/model/Model.cpp @@ -1060,39 +1060,20 @@ Model::_NotifyAuthorizationChanged() } +/*! This method will find the stored 'DepotInfo' that correlates to the + supplied 'url' and will invoke the mapper function in order to get a + replacement for the 'DepotInfo'. The 'url' is a unique identifier + for the repository that holds across mirrors. +*/ + void -Model::ForAllDepots(void (*func)(const DepotInfo& depot, void* context), +Model::ReplaceDepotByUrl(const BString& URL, DepotMapper* depotMapper, void* context) { for (int32 i = 0; i < fDepots.CountItems(); i++) { DepotInfo depotInfo = fDepots.ItemAtFast(i); - func(depotInfo, context); - } -} - -/*! This method will find the stored 'DepotInfo' that correlates to the - supplied 'url' or 'baseUrl' and will invoke the mapper function in - order to get a replacement for the 'DepotInfo'. The two URLs are - different. The 'url' is a unique identifier for the repository that - holds across mirrors. The 'baseUrl' is the URL stem that was used - to access the repository data in the first place. The 'baseUrl' is - a legacy construct that exists from a time where the identifying - 'url' was not being relayed properly. -*/ - -void -Model::ReplaceDepotByUrl( - const BString& URL, - const BString& baseURL, - // deprecated - DepotMapper* depotMapper, void* context) -{ - for (int32 i = 0; i < fDepots.CountItems(); i++) { - DepotInfo depotInfo = fDepots.ItemAtFast(i); - - if (RepositoryUrlUtils::EqualsOnUrlOrBaseUrl(URL, depotInfo.URL(), - baseURL, depotInfo.BaseURL())) { + if (RepositoryUrlUtils::EqualsNormalized(URL, depotInfo.URL())) { BAutolock locker(&fLock); fDepots.Replace(i, depotMapper->MapDepot(depotInfo, context)); } @@ -1100,53 +1081,6 @@ Model::ReplaceDepotByUrl( } -void -Model::ForAllPackages(PackageConsumer* packageConsumer, void* context) -{ - for (int32 i = 0; i < fDepots.CountItems(); i++) { - DepotInfo depotInfo = fDepots.ItemAtFast(i); - PackageList packages = depotInfo.Packages(); - for(int32 j = 0; j < packages.CountItems(); j++) { - const PackageInfoRef& packageInfoRef = packages.ItemAtFast(j); - - if (packageInfoRef != NULL) { - BAutolock locker(&fLock); - if (!packageConsumer->ConsumePackage(packageInfoRef, context)) - return; - } - } - } -} - - -void -Model::ForPackageByNameInDepot(const BString& depotName, - const BString& packageName, PackageConsumer* packageConsumer, void* context) -{ - int32 depotCount = fDepots.CountItems(); - - for (int32 i = 0; i < depotCount; i++) { - DepotInfo depotInfo = fDepots.ItemAtFast(i); - - if (depotInfo.Name() == depotName) { - int32 packageIndex = depotInfo.PackageIndexByName(packageName); - - if (-1 != packageIndex) { - PackageList packages = depotInfo.Packages(); - const PackageInfoRef& packageInfoRef = - packages.ItemAtFast(packageIndex); - - BAutolock locker(&fLock); - packageConsumer->ConsumePackage(packageInfoRef, - context); - } - - return; - } - } -} - - void Model::LogDepotsWithNoWebAppRepositoryCode() const { @@ -1158,8 +1092,8 @@ Model::LogDepotsWithNoWebAppRepositoryCode() const if (depot.WebAppRepositoryCode().Length() == 0) { printf("depot [%s]", depot.Name().String()); - if (depot.BaseURL().Length() > 0) - printf(" (%s)", depot.BaseURL().String()); + if (depot.URL().Length() > 0) + printf(" (%s)", depot.URL().String()); printf(" correlates with no repository in the haiku" "depot server system\n"); diff --git a/src/apps/haikudepot/model/Model.h b/src/apps/haikudepot/model/Model.h index 52c01b1d54..3799c408f3 100644 --- a/src/apps/haikudepot/model/Model.h +++ b/src/apps/haikudepot/model/Model.h @@ -9,9 +9,8 @@ #include #include -#include "AbstractServerProcess.h" +#include "AbstractProcess.h" #include "LocalIconStore.h" -#include "BulkLoadContext.h" #include "PackageInfo.h" #include "WebAppInterface.h" @@ -168,24 +167,9 @@ public: void ReplaceDepotByUrl( const BString& URL, - const BString& baseURL, DepotMapper* depotMapper, void* context); - void ForAllDepots( - void (*func)(const DepotInfo& depot, - void* context), - void* context); - - void ForAllPackages(PackageConsumer* packageConsumer, - void* context); - - void ForPackageByNameInDepot( - const BString& depotName, - const BString& packageName, - PackageConsumer* packageConsumer, - void* context); - status_t IconStoragePath(BPath& path) const; status_t DumpExportRepositoryDataPath(BPath& path) const; status_t DumpExportPkgDataPath(BPath& path, diff --git a/src/apps/haikudepot/model/PackageInfo.cpp b/src/apps/haikudepot/model/PackageInfo.cpp index 24551d5693..e0ba889dde 100644 --- a/src/apps/haikudepot/model/PackageInfo.cpp +++ b/src/apps/haikudepot/model/PackageInfo.cpp @@ -15,7 +15,6 @@ #include - // #pragma mark - UserInfo @@ -1036,13 +1035,40 @@ PackageInfo::_NotifyListenersImmediate(uint32 changes) } +// #pragma mark - Sorting Functions + + +/*! This function is used with the List class in order to facilitate fast + ordered inserting of packages. + */ + +static int32 +PackageCompare(const PackageInfoRef& p1, const PackageInfoRef& p2) +{ + return p1->Name().Compare(p2->Name()); +} + + +/*! This function is used with the List class in order to facilitate fast + searching of packages. + */ + +static int32 +PackageFixedNameCompare(const void* context, + const PackageInfoRef& package) +{ + const BString* packageName = static_cast(context); + return packageName->Compare(package->Name()); +} + + // #pragma mark - DepotInfo::DepotInfo() : fName(), - fPackages(), + fPackages(&PackageCompare, &PackageFixedNameCompare), fWebAppRepositoryCode() { } @@ -1051,7 +1077,7 @@ DepotInfo::DepotInfo() DepotInfo::DepotInfo(const BString& name) : fName(name), - fPackages(), + fPackages(&PackageCompare, &PackageFixedNameCompare), fWebAppRepositoryCode(), fWebAppRepositorySourceCode() { @@ -1064,7 +1090,6 @@ DepotInfo::DepotInfo(const DepotInfo& other) fPackages(other.fPackages), fWebAppRepositoryCode(other.fWebAppRepositoryCode), fWebAppRepositorySourceCode(other.fWebAppRepositorySourceCode), - fBaseURL(other.fBaseURL), fURL(other.fURL) { } @@ -1075,7 +1100,6 @@ DepotInfo::operator=(const DepotInfo& other) { fName = other.fName; fPackages = other.fPackages; - fBaseURL = other.fBaseURL; fURL = other.fURL; fWebAppRepositoryCode = other.fWebAppRepositoryCode; fWebAppRepositorySourceCode = other.fWebAppRepositorySourceCode; @@ -1098,12 +1122,6 @@ DepotInfo::operator!=(const DepotInfo& other) const } -static int32 PackageCompare(const PackageInfoRef& p1, const PackageInfoRef& p2) -{ - return p1->Name().Compare(p2->Name()); -} - - /*! This method will insert the package into the list of packages in order so that the list of packages remains in order. */ @@ -1111,23 +1129,14 @@ static int32 PackageCompare(const PackageInfoRef& p1, const PackageInfoRef& p2) bool DepotInfo::AddPackage(const PackageInfoRef& package) { - return fPackages.AddOrdered(package, &PackageCompare); -} - - -static int32 -PackageFixedNameCompare(const void* context, - const PackageInfoRef& package) -{ - const BString* packageName = static_cast(context); - return packageName->Compare(package->Name()); + return fPackages.Add(package); } int32 -DepotInfo::PackageIndexByName(const BString& packageName) +DepotInfo::PackageIndexByName(const BString& packageName) const { - return fPackages.BinarySearch(&packageName, &PackageFixedNameCompare); + return fPackages.Search(&packageName); } @@ -1142,8 +1151,6 @@ DepotInfo::SyncPackages(const PackageList& otherPackages) for (int32 j = packages.CountItems() - 1; j >= 0; j--) { const PackageInfoRef& package = packages.ItemAtFast(j); if (package->Name() == otherPackage->Name()) { -// printf("%s: found package: '%s'\n", fName.String(), -// package->Name().String()); package->SetState(otherPackage->State()); package->SetLocalFilePath(otherPackage->LocalFilePath()); package->SetSystemDependency( @@ -1169,13 +1176,6 @@ DepotInfo::SyncPackages(const PackageList& otherPackages) } -void -DepotInfo::SetBaseURL(const BString& baseURL) -{ - fBaseURL = baseURL; -} - - void DepotInfo::SetURL(const BString& URL) { diff --git a/src/apps/haikudepot/model/PackageInfo.h b/src/apps/haikudepot/model/PackageInfo.h index 28e98f33ff..263d052e4a 100644 --- a/src/apps/haikudepot/model/PackageInfo.h +++ b/src/apps/haikudepot/model/PackageInfo.h @@ -423,14 +423,11 @@ public: bool AddPackage(const PackageInfoRef& package); - int32 PackageIndexByName(const BString& packageName); + int32 PackageIndexByName(const BString& packageName) + const; void SyncPackages(const PackageList& packages); - void SetBaseURL(const BString& baseURL); - const BString& BaseURL() const - { return fBaseURL; } - void SetURL(const BString& URL); const BString& URL() const { return fURL; } @@ -449,9 +446,6 @@ private: PackageList fPackages; BString fWebAppRepositoryCode; BString fWebAppRepositorySourceCode; - BString fBaseURL; - // this is the URL at which the configured repository will be - // accessed to get data. BString fURL; // this is actually a unique identifier for the repository. }; diff --git a/src/apps/haikudepot/server/AbstractProcess.cpp b/src/apps/haikudepot/server/AbstractProcess.cpp new file mode 100644 index 0000000000..bc74cedf73 --- /dev/null +++ b/src/apps/haikudepot/server/AbstractProcess.cpp @@ -0,0 +1,155 @@ +/* + * Copyright 2018, Andrew Lindesay . + * All rights reserved. Distributed under the terms of the MIT License. + */ +#include "AbstractProcess.h" + +#include +#include +#include + +#include +#include +#include + +#include "HaikuDepotConstants.h" +#include "Logger.h" + + +AbstractProcess::AbstractProcess() + : + fLock(), + fListener(NULL), + fWasStopped(false), + fProcessState(PROCESS_INITIAL), + fErrorStatus(B_OK) +{ +} + + +AbstractProcess::~AbstractProcess() +{ +} + + +void +AbstractProcess::SetListener(AbstractProcessListener* listener) +{ + AutoLocker locker(&fLock); + fListener = listener; +} + + +status_t +AbstractProcess::Run() +{ + { + AutoLocker locker(&fLock); + + if (ProcessState() != PROCESS_INITIAL) { + printf("cannot start process as it is not idle"); + return B_NOT_ALLOWED; + } + + if (fWasStopped) { + printf("cannot start process as it was stopped"); + return B_CANCELED; + } + + fProcessState = PROCESS_RUNNING; + } + + status_t runResult = RunInternal(); + + if (runResult != B_OK) + printf("[%s] an error has arisen; %s\n", Name(), strerror(runResult)); + + AbstractProcessListener* listener; + + { + AutoLocker locker(&fLock); + fProcessState = PROCESS_COMPLETE; + fErrorStatus = runResult; + listener = fListener; + } + + // this process may be part of a larger bulk-load process and + // if so, the process orchestration needs to know when this + // process has completed. + if (listener != NULL) + listener->ProcessExited(); + + return runResult; +} + + +bool +AbstractProcess::WasStopped() +{ + AutoLocker locker(&fLock); + return fWasStopped; +} + + +status_t +AbstractProcess::ErrorStatus() +{ + AutoLocker locker(&fLock); + return fErrorStatus; +} + + +/*! This method will stop the process. The actual process may carry on to + perform some tidy-ups on its thread so this does not stop the thread or + change the state of the process; just indicates to the running thread that + it should stop. If it has not yet been started then it will be put into + finished state. +*/ + +status_t +AbstractProcess::Stop() +{ + status_t result = B_CANCELED; + AbstractProcessListener* listener = NULL; + + { + AutoLocker locker(&fLock); + + if (!fWasStopped) { + fWasStopped = true; + result = StopInternal(); + + if (fProcessState == PROCESS_INITIAL) { + listener = fListener; + fProcessState = PROCESS_COMPLETE; + } + } + } + + if (listener != NULL) + listener->ProcessExited(); + + return result; +} + + +status_t +AbstractProcess::StopInternal() +{ + return B_NOT_ALLOWED; +} + + +bool +AbstractProcess::IsRunning() +{ + return ProcessState() == PROCESS_RUNNING; +} + + +process_state +AbstractProcess::ProcessState() +{ + AutoLocker locker(&fLock); + return fProcessState; +} \ No newline at end of file diff --git a/src/apps/haikudepot/server/AbstractProcess.h b/src/apps/haikudepot/server/AbstractProcess.h new file mode 100644 index 0000000000..001ff8724e --- /dev/null +++ b/src/apps/haikudepot/server/AbstractProcess.h @@ -0,0 +1,65 @@ +/* + * Copyright 2018, Andrew Lindesay . + * All rights reserved. Distributed under the terms of the MIT License. + */ + + +#ifndef ABSTRACT_PROCESS_H +#define ABSTRACT_PROCESS_H + +#include +#include + +#include "StandardMetaData.h" +#include "Stoppable.h" + + +typedef enum process_state { + PROCESS_INITIAL = 1 << 0, + PROCESS_RUNNING = 1 << 1, + PROCESS_COMPLETE = 1 << 2 +} process_state; + + +/*! Clients are able to subclass from this 'interface' in order to accept + call-backs when a process has exited; either through success or through + failure. + */ + +class AbstractProcessListener { +public: + virtual void ProcessExited() = 0; +}; + + +/*! This is the superclass of all Processes. */ + +class AbstractProcess : public Stoppable { +public: + AbstractProcess(); + virtual ~AbstractProcess(); + + virtual const char* Name() const = 0; + virtual const char* Description() const = 0; + status_t Run(); + status_t Stop(); + status_t ErrorStatus(); + bool IsRunning(); + bool WasStopped(); + process_state ProcessState(); + void SetListener(AbstractProcessListener* listener); + +protected: + virtual status_t RunInternal() = 0; + virtual status_t StopInternal(); + +private: + BLocker fLock; + AbstractProcessListener* + fListener; + bool fWasStopped; + process_state fProcessState; + status_t fErrorStatus; +}; + +#endif // ABSTRACT_PROCESS_H diff --git a/src/apps/haikudepot/server/AbstractServerProcess.cpp b/src/apps/haikudepot/server/AbstractServerProcess.cpp index 3187476515..c6cd574af3 100644 --- a/src/apps/haikudepot/server/AbstractServerProcess.cpp +++ b/src/apps/haikudepot/server/AbstractServerProcess.cpp @@ -2,6 +2,8 @@ * Copyright 2017-2018, Andrew Lindesay . * All rights reserved. Distributed under the terms of the MIT License. */ + + #include "AbstractServerProcess.h" #include @@ -9,10 +11,8 @@ #include #include -#include #include #include -#include #include #include @@ -34,14 +34,9 @@ #define TIMEOUT_MICROSECONDS 3e+7 -AbstractServerProcess::AbstractServerProcess( - AbstractServerProcessListener* listener, uint32 options) +AbstractServerProcess::AbstractServerProcess(uint32 options) : - fLock(), - fListener(listener), - fWasStopped(false), - fProcessState(SERVER_PROCESS_INITIAL), - fErrorStatus(B_OK), + AbstractProcess(), fOptions(options), fRequest(NULL) { @@ -69,60 +64,6 @@ AbstractServerProcess::ShouldAttemptNetworkDownload(bool hasDataAlready) } -status_t -AbstractServerProcess::Run() -{ - { - BAutolock locker(&fLock); - - if (ProcessState() != SERVER_PROCESS_INITIAL) { - printf("cannot start server process as it is not idle"); - return B_NOT_ALLOWED; - } - - fProcessState = SERVER_PROCESS_RUNNING; - } - - SetErrorStatus(RunInternal()); - - SetProcessState(SERVER_PROCESS_COMPLETE); - - // this process may be part of a larger bulk-load process and - // if so, the process orchestration needs to know when this - // process has completed. - - if (fListener != NULL) - fListener->ServerProcessExited(); - - return ErrorStatus(); -} - - -bool -AbstractServerProcess::WasStopped() -{ - BAutolock locker(&fLock); - return fWasStopped; -} - - -status_t -AbstractServerProcess::ErrorStatus() -{ - BAutolock locker(&fLock); - return fErrorStatus; -} - - -status_t -AbstractServerProcess::Stop() -{ - BAutolock locker(&fLock); - fWasStopped = true; - return StopInternal(); -} - - status_t AbstractServerProcess::StopInternal() { @@ -130,41 +71,7 @@ AbstractServerProcess::StopInternal() return fRequest->Stop(); } - return B_NOT_ALLOWED; -} - - -bool -AbstractServerProcess::IsRunning() -{ - return ProcessState() == SERVER_PROCESS_RUNNING; -} - - -void -AbstractServerProcess::SetErrorStatus(status_t value) -{ - BAutolock locker(&fLock); - - if (fErrorStatus == B_OK) { - fErrorStatus = value; - } -} - - -void -AbstractServerProcess::SetProcessState(process_state value) -{ - BAutolock locker(&fLock); - fProcessState = value; -} - - -process_state -AbstractServerProcess::ProcessState() -{ - BAutolock locker(&fLock); - return fProcessState; + return AbstractProcess::StopInternal(); } @@ -174,7 +81,11 @@ AbstractServerProcess::IfModifiedSinceHeaderValue(BString& headerValue) const BPath metaDataPath; BString jsonPath; - GetStandardMetaDataPath(metaDataPath); + status_t result = GetStandardMetaDataPath(metaDataPath); + + if (result != B_OK) + return result; + GetStandardMetaDataJsonPath(jsonPath); return IfModifiedSinceHeaderValue(headerValue, metaDataPath, jsonPath); @@ -246,8 +157,8 @@ AbstractServerProcess::PopulateMetaData( } -bool -AbstractServerProcess::LooksLikeGzip(const char *pathStr) const +/* static */ bool +AbstractServerProcess::LooksLikeGzip(const char *pathStr) { int l = strlen(pathStr); return l > 4 && 0 == strncmp(&pathStr[l - 3], ".gz", 3); @@ -268,7 +179,7 @@ AbstractServerProcess::ParseJsonFromFileWithListener( FILE* file = fopen(pathStr, "rb"); if (file == NULL) { - fprintf(stderr, "unable to find the meta data file at [%s]\n", + printf("[%s] unable to find the meta data file at [%s]\n", Name(), path.Path()); return B_FILE_NOT_FOUND; } @@ -348,17 +259,18 @@ AbstractServerProcess::DownloadToLocalFile(const BPath& targetFilePath, return B_CANCELED; if (redirects > MAX_REDIRECTS) { - fprintf(stdout, "exceeded %d redirects --> failure\n", MAX_REDIRECTS); + printf("[%s] exceeded %d redirects --> failure\n", Name(), + MAX_REDIRECTS); return B_IO_ERROR; } if (failures > MAX_FAILURES) { - fprintf(stdout, "exceeded %d failures\n", MAX_FAILURES); + printf("[%s] exceeded %d failures\n", Name(), MAX_FAILURES); return B_IO_ERROR; } - fprintf(stdout, "[%s] will stream '%s' to [%s]\n", - Name(), url.UrlString().String(), targetFilePath.Path()); + printf("[%s] will stream '%s' to [%s]\n", Name(), url.UrlString().String(), + targetFilePath.Path()); ToFileUrlProtocolListener listener(targetFilePath, Name(), Logger::IsTraceEnabled()); diff --git a/src/apps/haikudepot/server/AbstractServerProcess.h b/src/apps/haikudepot/server/AbstractServerProcess.h index 11e7cf5602..12e7e7ee69 100644 --- a/src/apps/haikudepot/server/AbstractServerProcess.h +++ b/src/apps/haikudepot/server/AbstractServerProcess.h @@ -3,6 +3,7 @@ * All rights reserved. Distributed under the terms of the MIT License. */ + #ifndef ABSTRACT_SERVER_PROCESS_H #define ABSTRACT_SERVER_PROCESS_H @@ -11,54 +12,29 @@ #include #include +#include "AbstractProcess.h" #include "StandardMetaData.h" -#include "Stoppable.h" -typedef enum process_options { +typedef enum server_process_options { SERVER_PROCESS_NO_NETWORKING = 1 << 0, SERVER_PROCESS_PREFER_CACHE = 1 << 1, SERVER_PROCESS_DROP_CACHE = 1 << 2 -} process_options; +} server_process_options; -typedef enum process_state { - SERVER_PROCESS_INITIAL = 1, - SERVER_PROCESS_RUNNING = 2, - SERVER_PROCESS_COMPLETE = 3 -} process_state; +/*! This is the superclass of Processes that communicate with the Haiku Depot + Server (HDS) system. +*/ -/*! Clients are able to subclass from this 'interface' in order to accept - call-backs when a process has exited; either through success or through - failure. - */ - -class AbstractServerProcessListener { +class AbstractServerProcess : public AbstractProcess { public: - virtual void ServerProcessExited() = 0; -}; - - -class AbstractServerProcess : public Stoppable { -public: - AbstractServerProcess( - AbstractServerProcessListener* listener, - uint32 options); + AbstractServerProcess(uint32 options); virtual ~AbstractServerProcess(); - virtual const char* Name() = 0; - status_t Run(); - status_t Stop(); - status_t ErrorStatus(); - bool IsRunning(); - bool WasStopped(); - protected: - virtual status_t RunInternal() = 0; - virtual status_t StopInternal(); - - virtual void GetStandardMetaDataPath( + virtual status_t GetStandardMetaDataPath( BPath& path) const = 0; virtual void GetStandardMetaDataJsonPath( BString& jsonPath) const = 0; @@ -92,27 +68,20 @@ protected: static bool IsSuccess(status_t e); +protected: + virtual status_t StopInternal(); + private: - BLocker fLock; - AbstractServerProcessListener* - fListener; - bool fWasStopped; - process_state fProcessState; - status_t fErrorStatus; uint32 fOptions; BHttpRequest* fRequest; - process_state ProcessState(); - void SetErrorStatus(status_t value); - void SetProcessState(process_state value); - status_t DownloadToLocalFile( const BPath& targetFilePath, const BUrl& url, uint32 redirects, uint32 failures); - bool LooksLikeGzip(const char *pathStr) const; + static bool LooksLikeGzip(const char *pathStr); }; diff --git a/src/apps/haikudepot/server/AbstractSingleFileServerProcess.cpp b/src/apps/haikudepot/server/AbstractSingleFileServerProcess.cpp index b470ba2f30..1bd16f0e45 100644 --- a/src/apps/haikudepot/server/AbstractSingleFileServerProcess.cpp +++ b/src/apps/haikudepot/server/AbstractSingleFileServerProcess.cpp @@ -2,18 +2,21 @@ * Copyright 2017-2018, Andrew Lindesay . * All rights reserved. Distributed under the terms of the MIT License. */ + + #include "AbstractSingleFileServerProcess.h" #include "HaikuDepotConstants.h" #include "Logger.h" +#include "ServerHelper.h" #include "ServerSettings.h" #include "StorageUtils.h" AbstractSingleFileServerProcess::AbstractSingleFileServerProcess( - AbstractServerProcessListener* listener, uint32 options) + uint32 options) : - AbstractServerProcess(listener, options) + AbstractServerProcess(options) { } @@ -29,9 +32,13 @@ AbstractSingleFileServerProcess::RunInternal() if (Logger::IsInfoEnabled()) printf("[%s] will fetch data\n", Name()); - BPath localPath = LocalPath(); + BPath localPath; + status_t result = GetLocalPath(localPath); + + if (result != B_OK) + return result; + BString urlPathComponent = UrlPathComponent(); - status_t result = B_OK; if (IsSuccess(result) && HasOption(SERVER_PROCESS_DROP_CACHE)) result = DeleteLocalFile(localPath); @@ -48,6 +55,19 @@ AbstractSingleFileServerProcess::RunInternal() result = DownloadToLocalFileAtomically( localPath, ServerSettings::CreateFullUrl(urlPathComponent)); + + if (!IsSuccess(result)) { + if (hasData) { + printf("[%s] failed to update data, but have old data " + "anyway so carry on with that\n", Name()); + result = B_OK; + } else { + printf("[%s] failed to obtain data\n", Name()); + } + } else { + if (Logger::IsInfoEnabled()) + printf("[%s] did fetch data\n", Name()); + } } if (IsSuccess(result)) { @@ -61,11 +81,6 @@ AbstractSingleFileServerProcess::RunInternal() } if (IsSuccess(result)) { - if (Logger::IsInfoEnabled()) - printf("[%s] did fetch data\n", Name()); - - // now load the data in and process it. - printf("[%s] will process data\n", Name()); result = ProcessLocalData(); @@ -82,3 +97,9 @@ AbstractSingleFileServerProcess::RunInternal() return result; } + +status_t +AbstractSingleFileServerProcess::GetStandardMetaDataPath(BPath& path) const +{ + return GetLocalPath(path); +} \ No newline at end of file diff --git a/src/apps/haikudepot/server/AbstractSingleFileServerProcess.h b/src/apps/haikudepot/server/AbstractSingleFileServerProcess.h index d069d7bc0d..698261c394 100644 --- a/src/apps/haikudepot/server/AbstractSingleFileServerProcess.h +++ b/src/apps/haikudepot/server/AbstractSingleFileServerProcess.h @@ -1,8 +1,9 @@ /* - * Copyright 2017, Andrew Lindesay . + * Copyright 2017-2018, Andrew Lindesay . * All rights reserved. Distributed under the terms of the MIT License. */ + #ifndef ABSTRACT_SINGLE_FILE_SERVER_PROCESS_H #define ABSTRACT_SINGLE_FILE_SERVER_PROCESS_H @@ -12,9 +13,7 @@ class AbstractSingleFileServerProcess : public AbstractServerProcess { public: - AbstractSingleFileServerProcess( - AbstractServerProcessListener* listener, - uint32 options); + AbstractSingleFileServerProcess(uint32 options); virtual ~AbstractSingleFileServerProcess(); protected: @@ -24,7 +23,9 @@ protected: virtual BString UrlPathComponent() = 0; - virtual BPath& LocalPath() = 0; + virtual status_t GetLocalPath(BPath& path) const = 0; + + virtual status_t GetStandardMetaDataPath(BPath& path) const; }; #endif // ABSTRACT_SINGLE_FILE_SERVER_PROCESS_H \ No newline at end of file diff --git a/src/apps/haikudepot/server/BulkLoadContext.cpp b/src/apps/haikudepot/server/BulkLoadContext.cpp deleted file mode 100644 index ffa24302d7..0000000000 --- a/src/apps/haikudepot/server/BulkLoadContext.cpp +++ /dev/null @@ -1,131 +0,0 @@ -/* - * Copyright 2017, Andrew Lindesay . - * All rights reserved. Distributed under the terms of the MIT License. - */ -#include "BulkLoadContext.h" - - -BulkLoadContext::BulkLoadContext() - : - fState(BULK_LOAD_INITIAL), - fIconProcess(NULL), - fRepositoryProcess(NULL), - fPkgProcesses(new List()), - fProcessOptions(0) -{ -} - - -BulkLoadContext::~BulkLoadContext() -{ - StopAllProcesses(); - - if (fIconProcess != NULL) - delete fIconProcess; - - if (fRepositoryProcess != NULL) - delete fRepositoryProcess; - - int32 count = fPkgProcesses->CountItems(); - int32 i; - - for (i = 0; i < count; i++) - delete fPkgProcesses->ItemAt(i); - - delete fPkgProcesses; -} - - -bulk_load_state -BulkLoadContext::State() -{ - return fState; -} - - -void -BulkLoadContext::SetState(bulk_load_state value) -{ - fState = value; -} - - -void -BulkLoadContext::StopAllProcesses() -{ - if (fIconProcess != NULL) - fIconProcess->Stop(); - - if (fRepositoryProcess != NULL) - fRepositoryProcess->Stop(); - - int32 count = fPkgProcesses->CountItems(); - int32 i; - - for (i = 0; i < count; i++) - fPkgProcesses->ItemAt(i)->Stop(); -} - - -AbstractServerProcess* -BulkLoadContext::IconProcess() -{ - return fIconProcess; -} - - -void -BulkLoadContext::SetIconProcess(AbstractServerProcess* value) -{ - fIconProcess = value; -} - - -AbstractServerProcess* -BulkLoadContext::RepositoryProcess() -{ - return fRepositoryProcess; -} - - -void -BulkLoadContext::SetRepositoryProcess( - AbstractServerProcess* value) -{ - fRepositoryProcess = value; -} - - -int32 -BulkLoadContext::CountPkgProcesses() -{ - return fPkgProcesses->CountItems(); -} - - -AbstractServerProcess* -BulkLoadContext::PkgProcessAt(int32 index) -{ - return fPkgProcesses->ItemAt(index); -} - - -void -BulkLoadContext::AddPkgProcess(AbstractServerProcess *value) -{ - fPkgProcesses->Add(value); -} - - -void -BulkLoadContext::AddProcessOption(uint32 flag) -{ - fProcessOptions = fProcessOptions | flag; -} - - -uint32 -BulkLoadContext::ProcessOptions() -{ - return fProcessOptions; -} \ No newline at end of file diff --git a/src/apps/haikudepot/server/BulkLoadContext.h b/src/apps/haikudepot/server/BulkLoadContext.h deleted file mode 100644 index 04d8ce2c2f..0000000000 --- a/src/apps/haikudepot/server/BulkLoadContext.h +++ /dev/null @@ -1,66 +0,0 @@ -/* - * Copyright 2017, Andrew Lindesay . - * All rights reserved. Distributed under the terms of the MIT License. - */ -#ifndef BULK_LOAD_CONTEXT_H -#define BULK_LOAD_CONTEXT_H - -#include -#include -#include - -#include "AbstractServerProcess.h" -#include "List.h" - - -typedef enum bulk_load_state { - BULK_LOAD_INITIAL = 1, - BULK_LOAD_REPOSITORY_AND_REFERENCE = 2, - BULK_LOAD_PKGS_AND_ICONS = 3, - BULK_LOAD_COMPLETE = 4 -} bulk_load_state; - - -class BulkLoadContext { -public: - BulkLoadContext(); - virtual ~BulkLoadContext(); - - void StopAllProcesses(); - - bulk_load_state State(); - void SetState(bulk_load_state value); - - AbstractServerProcess* - IconProcess(); - void SetIconProcess(AbstractServerProcess* value); - - AbstractServerProcess* - RepositoryProcess(); - void SetRepositoryProcess( - AbstractServerProcess* value); - - int32 CountPkgProcesses(); - AbstractServerProcess* - PkgProcessAt(int32 index); - void AddPkgProcess(AbstractServerProcess *value); - - void AddProcessOption(uint32 flag); - uint32 ProcessOptions(); - -private: - bulk_load_state - fState; - - AbstractServerProcess* - fIconProcess; - AbstractServerProcess* - fRepositoryProcess; - List* - fPkgProcesses; - uint32 fProcessOptions; - -}; - - -#endif // BULK_LOAD_CONTEXT_H diff --git a/src/apps/haikudepot/server/BulkLoadStateMachine.cpp b/src/apps/haikudepot/server/BulkLoadStateMachine.cpp deleted file mode 100644 index c88964816b..0000000000 --- a/src/apps/haikudepot/server/BulkLoadStateMachine.cpp +++ /dev/null @@ -1,358 +0,0 @@ -/* - * Copyright 2017-2018, Andrew Lindesay . - * All rights reserved. Distributed under the terms of the MIT License. - */ -#include "BulkLoadStateMachine.h" - -#include - -#include "Logger.h" -#include "PkgDataUpdateProcess.h" -#include "RepositoryDataUpdateProcess.h" -#include "ServerIconExportUpdateProcess.h" -#include "ServerSettings.h" -#include "ServerHelper.h" - - -BulkLoadStateMachine::BulkLoadStateMachine(Model* model) - : - fBulkLoadContext(NULL), - fModel(model) -{ -} - - -BulkLoadStateMachine::~BulkLoadStateMachine() -{ - Stop(); -} - - -bool -BulkLoadStateMachine::IsRunning() -{ - BAutolock locker(&fLock); - return fBulkLoadContext != NULL; -} - - -/*! This gets invoked when one of the background processes has exited. */ - -void -BulkLoadStateMachine::ServerProcessExited() -{ - ContextPoll(); -} - - -static const char* bulk_load_state_name(bulk_load_state state) { - switch(state) { - case BULK_LOAD_INITIAL: - return "BULK_LOAD_INITIAL"; - case BULK_LOAD_REPOSITORY_AND_REFERENCE: - return "BULK_LOAD_REPOSITORY_AND_REFERENCE"; - case BULK_LOAD_PKGS_AND_ICONS: - return "BULK_LOAD_PKGS_AND_ICONS"; - case BULK_LOAD_COMPLETE: - return "BULK_LOAD_COMPLETE"; - default: - return "???"; - } -} - - -void -BulkLoadStateMachine::SetContextState(bulk_load_state state) -{ - if (Logger::IsDebugEnabled()) { - printf("bulk load - transition to state [%s]\n", - bulk_load_state_name(state)); - } - - fBulkLoadContext->SetState(state); -} - - -/*! Bulk loading data into the model can be considered to be a state - machine. This method is invoked each time that an event state - change happens. - */ - -void -BulkLoadStateMachine::ContextPoll() -{ - BAutolock locker(&fLock); - - if (Logger::IsDebugEnabled()) - printf("bulk load - context poll\n"); - - if (CanTransitionTo(BULK_LOAD_REPOSITORY_AND_REFERENCE)) { - SetContextState(BULK_LOAD_REPOSITORY_AND_REFERENCE); - InitiateBulkPopulateIcons(); - if (InitiateBulkRepositories() != B_OK) - ContextPoll(); - return; - } - - if (CanTransitionTo(BULK_LOAD_PKGS_AND_ICONS)) { - fModel->LogDepotsWithNoWebAppRepositoryCode(); - SetContextState(BULK_LOAD_PKGS_AND_ICONS); - InitiateBulkPopulatePackagesForAllDepots(); - return; - } - - if (CanTransitionTo(BULK_LOAD_COMPLETE)) { - SetContextState(BULK_LOAD_COMPLETE); - delete fBulkLoadContext; - fBulkLoadContext = NULL; - return; - } -} - - -bool -BulkLoadStateMachine::CanTransitionTo(bulk_load_state targetState) -{ - if (fBulkLoadContext != NULL) { - bulk_load_state existingState = fBulkLoadContext->State(); - - switch (targetState) { - case BULK_LOAD_INITIAL: - return false; - case BULK_LOAD_REPOSITORY_AND_REFERENCE: - return existingState == BULK_LOAD_INITIAL; - case BULK_LOAD_PKGS_AND_ICONS: - return (existingState == BULK_LOAD_REPOSITORY_AND_REFERENCE) - && ((fBulkLoadContext->RepositoryProcess() == NULL) - || !fBulkLoadContext->RepositoryProcess()->IsRunning()); - case BULK_LOAD_COMPLETE: - if ((existingState == BULK_LOAD_PKGS_AND_ICONS) - && ((fBulkLoadContext->IconProcess() == NULL) - || !fBulkLoadContext->IconProcess()->IsRunning())) { - int32 i; - - for (i = 0; i < fBulkLoadContext->CountPkgProcesses(); i++) { - AbstractServerProcess* process = - fBulkLoadContext->PkgProcessAt(i); - if (process->IsRunning()) - return false; - } - - return true; - } - break; - } - } - - return false; -} - - -void -BulkLoadStateMachine::StopAllProcesses() -{ - BAutolock locker(&fLock); - - if (fBulkLoadContext != NULL) { - if (NULL != fBulkLoadContext->IconProcess()) - fBulkLoadContext->IconProcess()->Stop(); - - if (NULL != fBulkLoadContext->RepositoryProcess()) - fBulkLoadContext->RepositoryProcess()->Stop(); - - int32 i; - - for(i = 0; i < fBulkLoadContext->CountPkgProcesses(); i++) { - AbstractServerProcess* serverProcess = - fBulkLoadContext->PkgProcessAt(i); - serverProcess->Stop(); - } - } -} - - -void -BulkLoadStateMachine::Start() -{ - if (Logger::IsInfoEnabled()) - printf("bulk load - start\n"); - - Stop(); - - { - BAutolock locker(&fLock); - - if (!IsRunning()) { - fBulkLoadContext = new BulkLoadContext(); - - if (ServerSettings::IsClientTooOld()) { - printf("bulk load proceeding without network communications " - "because the client is too old\n"); - fBulkLoadContext->AddProcessOption( - SERVER_PROCESS_NO_NETWORKING); - } - - if (!ServerHelper::IsNetworkAvailable()) { - fBulkLoadContext->AddProcessOption( - SERVER_PROCESS_NO_NETWORKING); - } - - if (ServerSettings::PreferCache()) - fBulkLoadContext->AddProcessOption(SERVER_PROCESS_PREFER_CACHE); - - if (ServerSettings::DropCache()) - fBulkLoadContext->AddProcessOption(SERVER_PROCESS_DROP_CACHE); - - ContextPoll(); - } - } -} - - -void -BulkLoadStateMachine::Stop() -{ - StopAllProcesses(); - - // spin lock to wait for the bulk-load processes to complete. - - while (IsRunning()) - snooze(500000); -} - - -/*! This method is the initial function that is invoked on starting a new - thread. It will start a server process that is part of the bulk-load. - */ - -status_t -BulkLoadStateMachine::StartProcess(void* cookie) -{ - AbstractServerProcess* process = - static_cast(cookie); - - if (Logger::IsInfoEnabled()) { - printf("bulk load - starting process [%s]\n", - process->Name()); - } - - process->Run(); - return B_OK; -} - - -status_t -BulkLoadStateMachine::InitiateServerProcess(AbstractServerProcess* process) -{ - if (Logger::IsInfoEnabled()) - printf("bulk load - initiating [%s]\n", process->Name()); - - thread_id tid = spawn_thread(&StartProcess, - process->Name(), B_NORMAL_PRIORITY, process); - - if (tid >= 0) { - resume_thread(tid); - return B_OK; - } - - return B_ERROR; -} - - -status_t -BulkLoadStateMachine::InitiateBulkRepositories() -{ - status_t result = B_OK; - BPath dataPath; - - fBulkLoadContext->SetRepositoryProcess(NULL); - result = fModel->DumpExportRepositoryDataPath(dataPath); - - if (result != B_OK) { - BAutolock locker(&fLock); - printf("unable to obtain the path for storing the repository data\n"); - ContextPoll(); - return B_ERROR; - } - - fBulkLoadContext->SetRepositoryProcess( - new RepositoryDataUpdateProcess(this, dataPath, fModel, - fBulkLoadContext->ProcessOptions())); - return InitiateServerProcess(fBulkLoadContext->RepositoryProcess()); -} - - -status_t -BulkLoadStateMachine::InitiateBulkPopulateIcons() -{ - BPath path; - - if (fModel->IconStoragePath(path) != B_OK) { - BAutolock locker(&fLock); - printf("unable to obtain the path for storing icons\n"); - ContextPoll(); - return B_ERROR; - } - - AbstractServerProcess *process = new ServerIconExportUpdateProcess( - this, path, fModel, fBulkLoadContext->ProcessOptions()); - fBulkLoadContext->SetIconProcess(process); - return InitiateServerProcess(process); -} - - -status_t -BulkLoadStateMachine::InitiateBulkPopulatePackagesForDepot( - const DepotInfo& depotInfo) -{ - BString repositorySourceCode = depotInfo.WebAppRepositorySourceCode(); - - if (repositorySourceCode.Length() == 0) { - printf("the depot [%s] has no repository source code\n", - depotInfo.Name().String()); - return B_ERROR; - } - - BPath repositorySourcePkgDataPath; - - if (fModel->DumpExportPkgDataPath(repositorySourcePkgDataPath, - repositorySourceCode) != B_OK) { - BAutolock locker(&fLock); - printf("unable to obtain the path for storing data for [%s]\n", - repositorySourceCode.String()); - ContextPoll(); - return B_ERROR; - } - - AbstractServerProcess *process = new PkgDataUpdateProcess( - this, repositorySourcePkgDataPath, fModel->PreferredLanguage(), - repositorySourceCode, depotInfo.Name(), fModel, - fBulkLoadContext->ProcessOptions()); - fBulkLoadContext->AddPkgProcess(process); - - return InitiateServerProcess(process); -} - - -// static -void -BulkLoadStateMachine::InitiatePopulatePackagesForDepotCallback( - const DepotInfo& depotInfo, void* context) -{ - BulkLoadStateMachine* stateMachine = - static_cast(context); - stateMachine->InitiateBulkPopulatePackagesForDepot(depotInfo); -} - - -void -BulkLoadStateMachine::InitiateBulkPopulatePackagesForAllDepots() -{ - fModel->ForAllDepots(&InitiatePopulatePackagesForDepotCallback, this); - - printf("did initiate populate package data for %" B_PRId32 " depots\n", - fBulkLoadContext->CountPkgProcesses()); - - if (0 == fBulkLoadContext->CountPkgProcesses()) - ContextPoll(); -} \ No newline at end of file diff --git a/src/apps/haikudepot/server/BulkLoadStateMachine.h b/src/apps/haikudepot/server/BulkLoadStateMachine.h deleted file mode 100644 index b1063e5d6d..0000000000 --- a/src/apps/haikudepot/server/BulkLoadStateMachine.h +++ /dev/null @@ -1,59 +0,0 @@ -/* - * Copyright 2017-2018, Andrew Lindesay . - * All rights reserved. Distributed under the terms of the MIT License. - */ -#ifndef BULK_LOAD_STATE_MACHINE_H -#define BULK_LOAD_STATE_MACHINE_H - -#include -#include -#include -#include - -#include "AbstractServerProcess.h" -#include "Model.h" - - -class BulkLoadStateMachine : public AbstractServerProcessListener { -public: - BulkLoadStateMachine(Model* model); - virtual ~BulkLoadStateMachine(); - - bool IsRunning(); - - void Start(); - void Stop(); - - void ServerProcessExited(); - -private: - static status_t StartProcess(void* cookie); - void ContextPoll(); - void SetContextState(bulk_load_state state); - void StopAllProcesses(); - - bool CanTransitionTo( - bulk_load_state targetState); - - static void InitiatePopulatePackagesForDepotCallback( - const DepotInfo& depotInfo, - void* context); - - status_t InitiateServerProcess( - AbstractServerProcess* process); - status_t InitiateBulkRepositories(); - status_t InitiateBulkPopulateIcons(); - status_t InitiateBulkPopulatePackagesForDepot( - const DepotInfo& depotInfo); - void InitiateBulkPopulatePackagesForAllDepots(); - - -private: - BLocker fLock; - BulkLoadContext* fBulkLoadContext; - Model* fModel; - -}; - - -#endif // BULK_LOAD_STATE_MACHINE_H diff --git a/src/apps/haikudepot/server/LocalPkgDataLoadProcess.cpp b/src/apps/haikudepot/server/LocalPkgDataLoadProcess.cpp new file mode 100644 index 0000000000..e9b0d3fba8 --- /dev/null +++ b/src/apps/haikudepot/server/LocalPkgDataLoadProcess.cpp @@ -0,0 +1,393 @@ +/* + * Copyright 2018, Andrew Lindesay . + * All rights reserved. Distributed under the terms of the MIT License. + */ + + +#include "LocalPkgDataLoadProcess.h" + +#include +#include + +#include +#include +#include +#include +#include +#include + +#include "AppUtils.h" +#include "HaikuDepotConstants.h" +#include "Logger.h" +#include "PackageInfo.h" +#include "PackageManager.h" +#include "RepositoryUrlUtils.h" + +#include +#include +#include +#include +#include "package/RepositoryCache.h" +#include +#include +#include + + +#undef B_TRANSLATION_CONTEXT +#define B_TRANSLATION_CONTEXT "LocalPkgDataLoadProcess" + + +using namespace BPackageKit; +using namespace BPackageKit::BManager::BPrivate; + + +typedef std::map PackageInfoMap; + + +/*! + \param packageInfoListener is assigned to each package model object. +*/ + +LocalPkgDataLoadProcess::LocalPkgDataLoadProcess( + PackageInfoListener* packageInfoListener, + Model *model, bool force) + : + AbstractProcess(), + fModel(model), + fForce(force), + fPackageInfoListener(packageInfoListener) +{ +} + + +LocalPkgDataLoadProcess::~LocalPkgDataLoadProcess() +{ +} + + +const char* +LocalPkgDataLoadProcess::Name() const +{ + return "LocalPkgDataLoadProcess"; +} + + +const char* +LocalPkgDataLoadProcess::Description() const +{ + return B_TRANSLATE("Reading repository data"); +} + + +/*! The contents of this method implementation have been 'lifted and shifted' + from MainWindow.cpp in order that the logic fits into the background + loading processes. The code needs to be broken up into methods with some + sort of a state object carrying the state of the process. As part of this, + better error handling and error reporting would also be advantageous. +*/ + +status_t +LocalPkgDataLoadProcess::RunInternal() +{ + if (Logger::IsDebugEnabled()) + printf("[%s] will refresh the package list\n", Name()); + + BPackageRoster roster; + BStringList repositoryNames; + + status_t result = roster.GetRepositoryNames(repositoryNames); + + if (result != B_OK) + return result; + + std::vector depots(repositoryNames.CountStrings()); + for (int32 i = 0; i < repositoryNames.CountStrings(); i++) { + const BString& repoName = repositoryNames.StringAt(i); + DepotInfo depotInfo = DepotInfo(repoName); + + BRepositoryConfig repoConfig; + status_t getRepositoryConfigStatus = roster.GetRepositoryConfig( + repoName, &repoConfig); + + if (getRepositoryConfigStatus == B_OK) { + depotInfo.SetURL(repoConfig.URL()); + + if (Logger::IsDebugEnabled()) { + printf("[%s] local repository [%s] info;\n" + " * url [%s]\n", Name(), repoName.String(), + repoConfig.URL().String()); + } + } else { + printf("[%s] unable to obtain the repository config for local " + "repository '%s'; %s\n", Name(), + repoName.String(), strerror(getRepositoryConfigStatus)); + } + + depots[i] = depotInfo; + } + + PackageManager manager(B_PACKAGE_INSTALLATION_LOCATION_HOME); + try { + manager.Init(PackageManager::B_ADD_INSTALLED_REPOSITORIES + | PackageManager::B_ADD_REMOTE_REPOSITORIES); + } catch (BException ex) { + BString message(B_TRANSLATE("An error occurred while " + "initializing the package manager: %message%")); + message.ReplaceFirst("%message%", ex.Message()); + _NotifyError(message.String()); + return B_ERROR; + } + + BObjectList packages; + result = manager.Solver()->FindPackages("", + BSolver::B_FIND_CASE_INSENSITIVE | BSolver::B_FIND_IN_NAME + | BSolver::B_FIND_IN_SUMMARY | BSolver::B_FIND_IN_DESCRIPTION + | BSolver::B_FIND_IN_PROVIDES, + packages); + if (result != B_OK) { + BString message(B_TRANSLATE("An error occurred while " + "obtaining the package list: %message%")); + message.ReplaceFirst("%message%", strerror(result)); + _NotifyError(message.String()); + return B_ERROR; + } + + if (packages.IsEmpty()) + return B_ERROR; + + PackageInfoMap foundPackages; + // if a given package is installed locally, we will potentially + // get back multiple entries, one for each local installation + // location, and one for each remote repository the package + // is available in. The above map is used to ensure that in such + // cases we consolidate the information, rather than displaying + // duplicates + PackageInfoMap remotePackages; + // any package that we find in a remote repository goes in this map. + // this is later used to discern which packages came from a local + // installation only, as those must be handled a bit differently + // upon uninstallation, since we'd no longer be able to pull them + // down remotely. + BStringList systemFlaggedPackages; + // any packages flagged as a system package are added to this list. + // such packages cannot be uninstalled, nor can any of their deps. + PackageInfoMap systemInstalledPackages; + // any packages installed in system are added to this list. + // This is later used for dependency resolution of the actual + // system packages in order to compute the list of protected + // dependencies indicated above. + + for (int32 i = 0; i < packages.CountItems(); i++) { + BSolverPackage* package = packages.ItemAt(i); + const BPackageInfo& repoPackageInfo = package->Info(); + const BString repositoryName = package->Repository()->Name(); + PackageInfoRef modelInfo; + PackageInfoMap::iterator it = foundPackages.find( + repoPackageInfo.Name()); + if (it != foundPackages.end()) + modelInfo.SetTo(it->second); + else { + // Add new package info + modelInfo.SetTo(new(std::nothrow) PackageInfo(repoPackageInfo), + true); + + if (modelInfo.Get() == NULL) + return B_ERROR; + + foundPackages[repoPackageInfo.Name()] = modelInfo; + } + + // The package list here considers those packages that are installed + // in the system as well as those that exist in remote repositories. + // It is better if the 'depot name' is from the remote repository + // because then it will be possible to perform a rating on it later. + + if (modelInfo->DepotName().IsEmpty() + || modelInfo->DepotName() == REPOSITORY_NAME_SYSTEM + || modelInfo->DepotName() == REPOSITORY_NAME_INSTALLED) { + modelInfo->SetDepotName(repositoryName); + } + + modelInfo->AddListener(fPackageInfoListener); + + BSolverRepository* repository = package->Repository(); + BPackageManager::RemoteRepository* remoteRepository = + dynamic_cast(repository); + + if (remoteRepository != NULL) { + + std::vector::iterator it; + + for (it = depots.begin(); it != depots.end(); it++) { + if (RepositoryUrlUtils::EqualsNormalized( + it->URL(), remoteRepository->Config().URL())) { + break; + } + } + + if (it == depots.end()) { + if (Logger::IsDebugEnabled()) { + printf("pkg [%s] repository [%s] not recognized" + " --> ignored\n", + modelInfo->Name().String(), repositoryName.String()); + } + } else { + it->AddPackage(modelInfo); + + if (Logger::IsTraceEnabled()) { + printf("pkg [%s] assigned to [%s]\n", + modelInfo->Name().String(), repositoryName.String()); + } + } + + remotePackages[modelInfo->Name()] = modelInfo; + } else { + if (repository == static_cast( + manager.SystemRepository())) { + modelInfo->AddInstallationLocation( + B_PACKAGE_INSTALLATION_LOCATION_SYSTEM); + if (!modelInfo->IsSystemPackage()) { + systemInstalledPackages[repoPackageInfo.FileName()] + = modelInfo; + } + } else if (repository == static_cast( + manager.HomeRepository())) { + modelInfo->AddInstallationLocation( + B_PACKAGE_INSTALLATION_LOCATION_HOME); + } + } + + if (modelInfo->IsSystemPackage()) + systemFlaggedPackages.Add(repoPackageInfo.FileName()); + } + + BAutolock lock(fModel->Lock()); + + if (fForce) + fModel->Clear(); + + // filter remote packages from the found list + // any packages remaining will be locally installed packages + // that weren't acquired from a repository + for (PackageInfoMap::iterator it = remotePackages.begin(); + it != remotePackages.end(); it++) { + foundPackages.erase(it->first); + } + + if (!foundPackages.empty()) { + BString repoName = B_TRANSLATE("Local"); + depots.push_back(DepotInfo(repoName)); + + for (PackageInfoMap::iterator it = foundPackages.begin(); + it != foundPackages.end(); ++it) { + depots.back().AddPackage(it->second); + } + } + + { + std::vector::iterator it; + + for (it = depots.begin(); it != depots.end(); it++) { + if (fModel->HasDepot(it->Name())) + fModel->SyncDepot(*it); + else + fModel->AddDepot(*it); + } + } + + // compute the OS package dependencies + try { + // create the solver + BSolver* solver; + status_t error = BSolver::Create(solver); + if (error != B_OK) + throw BFatalErrorException(error, "Failed to create solver."); + + ObjectDeleter solverDeleter(solver); + BPath systemPath; + error = find_directory(B_SYSTEM_PACKAGES_DIRECTORY, &systemPath); + if (error != B_OK) { + throw BFatalErrorException(error, + "Unable to retrieve system packages directory."); + } + + // add the "installed" repository with the given packages + BSolverRepository installedRepository; + { + BRepositoryBuilder installedRepositoryBuilder(installedRepository, + REPOSITORY_NAME_INSTALLED); + for (int32 i = 0; i < systemFlaggedPackages.CountStrings(); i++) { + BPath packagePath(systemPath); + packagePath.Append(systemFlaggedPackages.StringAt(i)); + installedRepositoryBuilder.AddPackage(packagePath.Path()); + } + installedRepositoryBuilder.AddToSolver(solver, true); + } + + // add system repository + BSolverRepository systemRepository; + { + BRepositoryBuilder systemRepositoryBuilder(systemRepository, + REPOSITORY_NAME_SYSTEM); + for (PackageInfoMap::iterator it = systemInstalledPackages.begin(); + it != systemInstalledPackages.end(); it++) { + BPath packagePath(systemPath); + packagePath.Append(it->first); + systemRepositoryBuilder.AddPackage(packagePath.Path()); + } + systemRepositoryBuilder.AddToSolver(solver, false); + } + + // solve + error = solver->VerifyInstallation(); + if (error != B_OK) { + throw BFatalErrorException(error, "Failed to compute packages to " + "install."); + } + + BSolverResult solverResult; + error = solver->GetResult(solverResult); + if (error != B_OK) { + throw BFatalErrorException(error, "Failed to retrieve system " + "package dependency list."); + } + + for (int32 i = 0; const BSolverResultElement* element + = solverResult.ElementAt(i); i++) { + BSolverPackage* package = element->Package(); + if (element->Type() == BSolverResultElement::B_TYPE_INSTALL) { + PackageInfoMap::iterator it = systemInstalledPackages.find( + package->Info().FileName()); + if (it != systemInstalledPackages.end()) + it->second->SetSystemDependency(true); + } + } + } catch (BFatalErrorException ex) { + printf("Fatal exception occurred while resolving system dependencies: " + "%s, details: %s\n", strerror(ex.Error()), ex.Details().String()); + } catch (BNothingToDoException) { + // do nothing + } catch (BException ex) { + printf("Exception occurred while resolving system dependencies: %s\n", + ex.Message().String()); + } catch (...) { + printf("Unknown exception occurred while resolving system " + "dependencies.\n"); + } + + if (Logger::IsDebugEnabled()) + printf("did refresh the package list\n"); + + return B_OK; +} + + +void +LocalPkgDataLoadProcess::_NotifyError(const BString& messageText) const +{ + printf("an error has arisen loading data of packages from local : %s\n", + messageText.String()); + AppUtils::NotifySimpleError( + B_TRANSLATE("Local Repository Load Error"), + messageText); +} \ No newline at end of file diff --git a/src/apps/haikudepot/server/LocalPkgDataLoadProcess.h b/src/apps/haikudepot/server/LocalPkgDataLoadProcess.h new file mode 100644 index 0000000000..721439f2d2 --- /dev/null +++ b/src/apps/haikudepot/server/LocalPkgDataLoadProcess.h @@ -0,0 +1,55 @@ +/* + * Copyright 2018, Andrew Lindesay . + * All rights reserved. Distributed under the terms of the MIT License. + */ + + +#ifndef LOCAL_PKG_DATA_LOAD_PROCESS__H +#define LOCAL_PKG_DATA_LOAD_PROCESS__H + + +#include "AbstractProcess.h" + +#include "Model.h" +#include "PackageInfo.h" + +#include +#include +#include +#include + +#include + + +class PkgDataLoadState; + + +/*! This process will take the data from the locally stored repositories (HPKR) + and will extract the packages. The packages are then loaded into the + HaikuDepot 'Model'. +*/ + +class LocalPkgDataLoadProcess : public AbstractProcess { +public: + LocalPkgDataLoadProcess( + PackageInfoListener* packageInfoListener, + Model *model, bool force = false); + virtual ~LocalPkgDataLoadProcess(); + + const char* Name() const; + const char* Description() const; + +protected: + virtual status_t RunInternal(); + +private: + void _NotifyError(const BString& messageText) const; + +private: + Model* fModel; + bool fForce; + PackageInfoListener* + fPackageInfoListener; +}; + +#endif // LOCAL_PKG_DATA_LOAD_PROCESS__H diff --git a/src/apps/haikudepot/server/LocalRepositoryUpdateProcess.cpp b/src/apps/haikudepot/server/LocalRepositoryUpdateProcess.cpp new file mode 100644 index 0000000000..c0bfb5128f --- /dev/null +++ b/src/apps/haikudepot/server/LocalRepositoryUpdateProcess.cpp @@ -0,0 +1,156 @@ +/* + * Copyright 2018, Andrew Lindesay . + * All rights reserved. Distributed under the terms of the MIT License. + */ + + +#include "LocalRepositoryUpdateProcess.h" + +#include +#include +#include +#include + +#include +#include +#include +#include + +#include "AppUtils.h" +#include "DecisionProvider.h" +#include "JobStateListener.h" +#include "Logger.h" +#include "HaikuDepotConstants.h" + + +#undef B_TRANSLATION_CONTEXT +#define B_TRANSLATION_CONTEXT "LocalRepositoryUpdateProcess" + + +using namespace BPackageKit; +using namespace BPackageKit::BManager::BPrivate; + + +LocalRepositoryUpdateProcess::LocalRepositoryUpdateProcess( + Model *model, bool force) + : + AbstractProcess(), + fModel(model), + fForce(force) +{ +} + + +LocalRepositoryUpdateProcess::~LocalRepositoryUpdateProcess() +{ +} + + +const char* +LocalRepositoryUpdateProcess::Name() const +{ + return "LocalRepositoryUpdateProcess"; +} + + +const char* +LocalRepositoryUpdateProcess::Description() const +{ + return B_TRANSLATE("Fetching remote repository data"); +} + + +status_t +LocalRepositoryUpdateProcess::RunInternal() +{ + BPackageRoster roster; + BStringList repoNames; + + if (Logger::IsInfoEnabled()) { + printf("[%s] will update local repositories\n", Name()); + } + + status_t result = roster.GetRepositoryNames(repoNames); + + if (result == B_OK) { + DecisionProvider decisionProvider; + JobStateListener listener; + BContext context(decisionProvider, listener); + BRepositoryCache cache; + + for ( + int32 i = 0; + result == B_OK && i < repoNames.CountStrings() && !WasStopped(); + ++i) { + result = _RunForRepositoryName(repoNames.StringAt(i), context, + roster, &cache); + } + } else { + _NotifyError(strerror(result)); + result = B_ERROR; + } + + if (result == B_OK && Logger::IsInfoEnabled()) { + printf("[%s] did update %" B_PRIi32 " local repositories\n", + Name(), repoNames.CountStrings()); + } + + return result; +} + +status_t +LocalRepositoryUpdateProcess::_RunForRepositoryName(const BString& repoName, + BPackageKit::BContext& context, BPackageKit::BPackageRoster& roster, + BPackageKit::BRepositoryCache* cache) +{ + status_t result = B_ERROR; + BRepositoryConfig repoConfig; + result = roster.GetRepositoryConfig(repoName, &repoConfig); + if (result == B_OK) { + if (roster.GetRepositoryCache(repoName, cache) != B_OK || fForce) { + try { + BRefreshRepositoryRequest refreshRequest(context, repoConfig); + result = refreshRequest.Process(); + result = B_OK; + } catch (BFatalErrorException ex) { + _NotifyError(ex.Message(), ex.Details()); + } catch (BException ex) { + _NotifyError(ex.Message()); + } + } + } else { + _NotifyError(strerror(result)); + } + + return result; +} + + +void +LocalRepositoryUpdateProcess::_NotifyError(const BString& error) const +{ + _NotifyError(error, ""); +} + + +void +LocalRepositoryUpdateProcess::_NotifyError(const BString& error, + const BString& details) const +{ + printf("an error has arisen updating the local repositories : %s\n", + error.String()); + + BString alertText(B_TRANSLATE("An error occurred while refreshing the " + "repository: %error%")); + alertText.ReplaceFirst("%error%", error); + + if (!details.IsEmpty()) { + alertText.Append(" ("); + alertText.Append(details); + alertText.Append(")"); + } + + AppUtils::NotifySimpleError( + B_TRANSLATE("Repository Update Error"), + alertText); +} \ No newline at end of file diff --git a/src/apps/haikudepot/server/LocalRepositoryUpdateProcess.h b/src/apps/haikudepot/server/LocalRepositoryUpdateProcess.h new file mode 100644 index 0000000000..0062af6df2 --- /dev/null +++ b/src/apps/haikudepot/server/LocalRepositoryUpdateProcess.h @@ -0,0 +1,59 @@ +/* + * Copyright 2018, Andrew Lindesay . + * All rights reserved. Distributed under the terms of the MIT License. + */ + + +#ifndef LOCAL_REPOSITORY_UPDATE_PROCESS__H +#define LOCAL_REPOSITORY_UPDATE_PROCESS__H + + +#include "AbstractProcess.h" + +#include +#include +#include +#include + +#include +#include +#include + +#include "Model.h" +#include "PackageInfo.h" + + +/*! This process is intended to, for each repository configured on the host, + retrieve an updated set of data for the repository from the remote + repository site. This is typically HPKR data copied over to the local + machine. From there, a latter process will process this data by using the + facilities of the Package Kit. +*/ + +class LocalRepositoryUpdateProcess : public AbstractProcess { +public: + LocalRepositoryUpdateProcess( + Model *model, bool force = false); + virtual ~LocalRepositoryUpdateProcess(); + + const char* Name() const; + const char* Description() const; + +protected: + virtual status_t RunInternal(); + +private: + status_t _RunForRepositoryName(const BString& repoName, + BPackageKit::BContext& context, + BPackageKit::BPackageRoster& roster, + BPackageKit::BRepositoryCache* cache); + void _NotifyError(const BString& error) const; + void _NotifyError(const BString& error, + const BString& details) const; + +private: + Model* fModel; + bool fForce; +}; + +#endif // LOCAL_REPOSITORY_UPDATE_PROCESS__H diff --git a/src/apps/haikudepot/server/ProcessCoordinator.cpp b/src/apps/haikudepot/server/ProcessCoordinator.cpp new file mode 100644 index 0000000000..563c4e87fb --- /dev/null +++ b/src/apps/haikudepot/server/ProcessCoordinator.cpp @@ -0,0 +1,326 @@ +/* + * Copyright 2018, Andrew Lindesay . + * All rights reserved. Distributed under the terms of the MIT License. + */ + + +#include "ProcessCoordinator.h" + +#include +#include +#include + +#include "Logger.h" + + +#undef B_TRANSLATION_CONTEXT +#define B_TRANSLATION_CONTEXT "ProcessCoordinator" + + +// #pragma mark - ProcessCoordinatorState implementation + + +ProcessCoordinatorState::ProcessCoordinatorState( + const ProcessCoordinator* processCoordinator, float progress, + const BString& message, bool isRunning, status_t errorStatus) + : + fProcessCoordinator(processCoordinator), + fProgress(progress), + fMessage(message), + fIsRunning(isRunning), + fErrorStatus(errorStatus) +{ +} + + +ProcessCoordinatorState::~ProcessCoordinatorState() +{ +} + + +const ProcessCoordinator* +ProcessCoordinatorState::Coordinator() const +{ + return fProcessCoordinator; +} + + +float +ProcessCoordinatorState::Progress() const +{ + return fProgress; +} + + +BString +ProcessCoordinatorState::Message() const +{ + return fMessage; +} + + +bool +ProcessCoordinatorState::IsRunning() const +{ + return fIsRunning; +} + + +status_t +ProcessCoordinatorState::ErrorStatus() const +{ + return fErrorStatus; +} + + +// #pragma mark - ProcessCoordinator implementation + + +ProcessCoordinator::ProcessCoordinator(ProcessCoordinatorListener* listener) + : + fListener(listener), + fWasStopped(false) +{ +} + + +ProcessCoordinator::~ProcessCoordinator() +{ + AutoLocker locker(&fLock); + for (int32 i = 0; i < fNodes.CountItems(); i++) { + ProcessNode* node = fNodes.ItemAt(i); + node->Process()->SetListener(NULL); + delete node; + } +} + + +void +ProcessCoordinator::AddNode(ProcessNode* node) +{ + AutoLocker locker(&fLock); + fNodes.Add(node); + node->Process()->SetListener(this); +} + + +void +ProcessCoordinator::ProcessExited() +{ + _CoordinateAndCallListener(); +} + + +bool +ProcessCoordinator::IsRunning() +{ + AutoLocker locker(&fLock); + for (int32 i = 0; i < fNodes.CountItems(); i++) { + if (_IsRunning(fNodes.ItemAt(i))) + return true; + } + + return false; +} + + +void +ProcessCoordinator::Start() +{ + _CoordinateAndCallListener(); +} + + +void +ProcessCoordinator::Stop() +{ + AutoLocker locker(&fLock); + if (!fWasStopped) { + fWasStopped = true; + printf("[Coordinator] will stop process coordinator\n"); + for (int32 i = 0; i < fNodes.CountItems(); i++) { + ProcessNode* node = fNodes.ItemAt(i); + printf("[%s] stopping process", node->Process()->Name()); + if (node->Process()->ErrorStatus() != B_OK) + printf(" (error)\n"); + printf("\n"); + node->StopProcess(); + } + } +} + + +status_t +ProcessCoordinator::ErrorStatus() +{ + AutoLocker locker(&fLock); + for (int32 i = 0; i < fNodes.CountItems(); i++) { + status_t result = fNodes.ItemAt(i)->Process()->ErrorStatus(); + + if (result != B_OK) + return result; + } + + return B_OK; +} + + +float +ProcessCoordinator::Progress() +{ + AutoLocker locker(&fLock); + if (!fWasStopped) + return ((float) _CountNodesCompleted()) / ((float) fNodes.CountItems()); + return 0.0f; +} + + +BString +ProcessCoordinator::_CreateStatusMessage() +{ + // work through the nodes and take a description from the first one. If + // there are others present then use a 'plus X others' suffix. Go backwards + // through the processes so that the most recent activity is shown first. + + BString firstProcessDescription; + uint32 additionalRunningProcesses = 0; + + for (int32 i = fNodes.CountItems() - 1; i >= 0; i--) { + AbstractProcess* process = fNodes.ItemAt(i)->Process(); + + if (process->ProcessState() == PROCESS_RUNNING) { + if (firstProcessDescription.IsEmpty()) { + firstProcessDescription = process->Description(); + } else { + additionalRunningProcesses++; + } + } + } + + if (firstProcessDescription.IsEmpty()) + return "???"; + + if (additionalRunningProcesses == 0) + return firstProcessDescription; + + static BStringFormat format(B_TRANSLATE( + "%FIRST_PROCESS_DESCRIPTION% +" + "{0, plural, one{# process} other{# processes}}")); + BString result; + format.Format(result, additionalRunningProcesses); + result.ReplaceAll("%FIRST_PROCESS_DESCRIPTION%", firstProcessDescription); + + return result; +} + + +/*! This method assumes that a lock is held on the coordinator. */ + +ProcessCoordinatorState +ProcessCoordinator::_CreateStatus() +{ + return ProcessCoordinatorState( + this, Progress(), _CreateStatusMessage(), IsRunning(), ErrorStatus()); +} + + +void +ProcessCoordinator::_CoordinateAndCallListener() +{ + ProcessCoordinatorState state = _Coordinate(); + + if (fListener != NULL) + fListener->CoordinatorChanged(state); +} + + +ProcessCoordinatorState +ProcessCoordinator::_Coordinate() +{ + if (Logger::IsTraceEnabled()) + printf("[Coordinator] will coordinate nodes\n"); + + AutoLocker locker(&fLock); + + _StopSuccessorNodesToErroredOrStoppedNodes(); + + // go through the nodes and find those that are still to be run and + // for which the preconditions are met to start. + for (int32 i = 0; i < fNodes.CountItems(); i++) { + ProcessNode* node = fNodes.ItemAt(i); + + if (node->Process()->ProcessState() == PROCESS_INITIAL) { + if (node->AllPredecessorsComplete()) + node->StartProcess(); + else { + if (Logger::IsTraceEnabled()) { + printf("[Coordinator] all predecessors not complete -> " + "[%s] not started\n", node->Process()->Name()); + } + } + } else { + if (Logger::IsTraceEnabled()) { + printf("[Coordinator] process [%s] running or complete\n", + node->Process()->Name()); + } + } + } + + return _CreateStatus(); +} + + +/*! This method assumes that a lock is held on the coordinator. */ + +void +ProcessCoordinator::_StopSuccessorNodesToErroredOrStoppedNodes() +{ + for (int32 i = 0; i < fNodes.CountItems(); i++) { + ProcessNode* node = fNodes.ItemAt(i); + AbstractProcess* process = node->Process(); + + if (process->WasStopped() || process->ErrorStatus() != B_OK) + _StopSuccessorNodes(node); + } +} + + +/*! This method assumes that a lock is held on the coordinator. */ + +void +ProcessCoordinator::_StopSuccessorNodes(ProcessNode* predecessorNode) +{ + for (int32 i = 0; i < predecessorNode->CountSuccessors(); i++) { + ProcessNode* node = predecessorNode->SuccessorAt(i); + AbstractProcess* process = node->Process(); + + if (process->ProcessState() == PROCESS_INITIAL) { + if (Logger::IsDebugEnabled()) { + printf("[Coordinator] [%s] (failed) --> [%s] (stopping)\n", + predecessorNode->Process()->Name(), process->Name()); + } + node->StopProcess(); + _StopSuccessorNodes(node); + } + } +} + + +bool +ProcessCoordinator::_IsRunning(ProcessNode* node) +{ + return node->Process()->ProcessState() != PROCESS_COMPLETE; +} + + +int32 +ProcessCoordinator::_CountNodesCompleted() +{ + int32 nodesCompleted = 0; + for (int32 i = 0; i < fNodes.CountItems(); i++) { + AbstractProcess *process = fNodes.ItemAt(i)->Process(); + if (process->ProcessState() == PROCESS_COMPLETE) + nodesCompleted++; + } + return nodesCompleted; +} diff --git a/src/apps/haikudepot/server/ProcessCoordinator.h b/src/apps/haikudepot/server/ProcessCoordinator.h new file mode 100644 index 0000000000..c0a9d32fce --- /dev/null +++ b/src/apps/haikudepot/server/ProcessCoordinator.h @@ -0,0 +1,115 @@ +/* + * Copyright 2018, Andrew Lindesay . + * All rights reserved. Distributed under the terms of the MIT License. + */ + + +#ifndef PROCESS_COORDINATOR_H +#define PROCESS_COORDINATOR_H + +#include "ProcessCoordinator.h" + +#include "AbstractProcess.h" +#include "ProcessNode.h" +#include "List.h" + + +class ProcessCoordinator; + + +/*! This class carries the state of the current process coordinator so that + it can be dealt with atomically without having call back to the coordinator. +*/ + +class ProcessCoordinatorState { +public: + ProcessCoordinatorState( + const ProcessCoordinator* + processCoordinator, + float progress, const BString& message, + bool isRunning, status_t errorStatus); + virtual ~ProcessCoordinatorState(); + + const ProcessCoordinator* Coordinator() const; + float Progress() const; + BString Message() const; + bool IsRunning() const; + status_t ErrorStatus() const; + +private: + const ProcessCoordinator* fProcessCoordinator; + float fProgress; + BString fMessage; + bool fIsRunning; + status_t fErrorStatus; +}; + + +/*! Clients are able to subclass from this 'interface' in order to accept + call-backs when a coordinator has exited; either through failure, + stopping or completion. +*/ + +class ProcessCoordinatorListener { +public: + +/*! Signals to the listener that the coordinator has changed in some way - + for example, a process has started or stopped or even that the whole + coordinator has finished. +*/ + + virtual void CoordinatorChanged( + ProcessCoordinatorState& + processCoordinatorState) = 0; + +}; + + +/*! It is possible to create a number of ProcessNodes (themselves associated + with AbstractProcess-s) that may have dependencies (predecessors and + successors) and then an instance of this class is able to coordinate the + list of ProcessNode-s so that they are all completed in the correct order. +*/ + +class ProcessCoordinator : public AbstractProcessListener { +public: + ProcessCoordinator( + ProcessCoordinatorListener* listener); + virtual ~ProcessCoordinator(); + + void AddNode(ProcessNode* nodes); + + void ProcessExited(); + // AbstractProcessListener + + bool IsRunning(); + + void Start(); + void Stop(); + + status_t ErrorStatus(); + + float Progress(); + +private: + bool _IsRunning(ProcessNode* node); + void _CoordinateAndCallListener(); + ProcessCoordinatorState + _Coordinate(); + ProcessCoordinatorState + _CreateStatus(); + BString _CreateStatusMessage(); + int32 _CountNodesCompleted(); + void _StopSuccessorNodesToErroredOrStoppedNodes(); + void _StopSuccessorNodes(ProcessNode* node); + + BLocker fLock; + List + fNodes; + ProcessCoordinatorListener* + fListener; + bool fWasStopped; +}; + + +#endif // PROCESS_COORDINATOR_H diff --git a/src/apps/haikudepot/server/ProcessCoordinatorFactory.cpp b/src/apps/haikudepot/server/ProcessCoordinatorFactory.cpp new file mode 100644 index 0000000000..f5a544c524 --- /dev/null +++ b/src/apps/haikudepot/server/ProcessCoordinatorFactory.cpp @@ -0,0 +1,111 @@ +/* + * Copyright 2018, Andrew Lindesay . + * All rights reserved. Distributed under the terms of the MIT License. + */ + + +#include "ProcessCoordinatorFactory.h" + +#include + +#include +#include + +#include "AbstractServerProcess.h" +#include "LocalPkgDataLoadProcess.h" +#include "LocalRepositoryUpdateProcess.h" +#include "Model.h" +#include "PackageInfoListener.h" +#include "ProcessCoordinator.h" +#include "ProcessNode.h" +#include "ServerHelper.h" +#include "ServerIconExportUpdateProcess.h" +#include "ServerPkgDataUpdateProcess.h" +#include "ServerRepositoryDataUpdateProcess.h" +#include "ServerSettings.h" + + +using namespace BPackageKit; + + +/* static */ ProcessCoordinator* +ProcessCoordinatorFactory::CreateBulkLoadCoordinator( + PackageInfoListener *packageInfoListener, + ProcessCoordinatorListener* processCoordinatorListener, + Model* model, bool forceLocalUpdate) +{ + uint32 serverProcessOptions = _CalculateServerProcessOptions(); + BAutolock locker(model->Lock()); + ProcessCoordinator* processCoordinator = new ProcessCoordinator( + processCoordinatorListener); + + ProcessNode *localRepositoryUpdate = + new ProcessNode(new LocalRepositoryUpdateProcess(model, + forceLocalUpdate)); + processCoordinator->AddNode(localRepositoryUpdate); + + ProcessNode *localPkgDataLoad = + new ProcessNode(new LocalPkgDataLoadProcess( + packageInfoListener, model, forceLocalUpdate)); + localPkgDataLoad->AddPredecessor(localRepositoryUpdate); + processCoordinator->AddNode(localPkgDataLoad); + + ProcessNode *serverIconExportUpdate = + new ProcessNode(new ServerIconExportUpdateProcess(model, + serverProcessOptions)); + serverIconExportUpdate->AddPredecessor(localPkgDataLoad); + processCoordinator->AddNode(serverIconExportUpdate); + + ProcessNode *serverRepositoryDataUpdate = + new ProcessNode(new ServerRepositoryDataUpdateProcess(model, + serverProcessOptions)); + serverRepositoryDataUpdate->AddPredecessor(localPkgDataLoad); + processCoordinator->AddNode(serverRepositoryDataUpdate); + + // create a process for each of the repositories that are configured on the + // local system. Later, only those that have a web-app repository server + // code will be actually processed, but this means that the creation of the + // 'processes' does not need to be dynamic as the process coordinator runs. + + BPackageRoster roster; + BStringList repoNames; + status_t repoNamesResult = roster.GetRepositoryNames(repoNames); + + if (repoNamesResult == B_OK) { + for (int32 i = 0; i < repoNames.CountStrings(); i++) { + ProcessNode* processNode = new ProcessNode( + new ServerPkgDataUpdateProcess(model->PreferredLanguage(), + repoNames.StringAt(i), model, serverProcessOptions)); + processNode->AddPredecessor(serverRepositoryDataUpdate); + processCoordinator->AddNode(processNode); + } + } else { + printf("a problem has arisen getting the repository names.\n"); + } + + return processCoordinator; +} + + +/* static */ uint32 +ProcessCoordinatorFactory::_CalculateServerProcessOptions() +{ + uint32 processOptions = 0; + + if (ServerSettings::IsClientTooOld()) { + printf("bulk load proceeding without network communications " + "because the client is too old\n"); + processOptions |= SERVER_PROCESS_NO_NETWORKING; + } + + if (!ServerHelper::IsNetworkAvailable()) + processOptions |= SERVER_PROCESS_NO_NETWORKING; + + if (ServerSettings::PreferCache()) + processOptions |= SERVER_PROCESS_PREFER_CACHE; + + if (ServerSettings::DropCache()) + processOptions |= SERVER_PROCESS_DROP_CACHE; + + return processOptions; +} diff --git a/src/apps/haikudepot/server/ProcessCoordinatorFactory.h b/src/apps/haikudepot/server/ProcessCoordinatorFactory.h new file mode 100644 index 0000000000..e3ba0b580c --- /dev/null +++ b/src/apps/haikudepot/server/ProcessCoordinatorFactory.h @@ -0,0 +1,33 @@ +/* + * Copyright 2018, Andrew Lindesay . + * All rights reserved. Distributed under the terms of the MIT License. + */ + + +#ifndef PROCESS_COORDINATOR_FACTORY_H +#define PROCESS_COORDINATOR_FACTORY_H + +#include + +class Model; +class PackageInfoListener; +class ProcessCoordinator; +class ProcessCoordinatorListener; + +/*! This class is able to create ProcessCoordinators that are loaded-up with + Processes that together complete some larger job. +*/ + +class ProcessCoordinatorFactory { +public: + static ProcessCoordinator* CreateBulkLoadCoordinator( + PackageInfoListener *packageInfoListener, + ProcessCoordinatorListener* + processCoordinatorListener, + Model* model, bool forceLocalUpdate); +private: + static uint32 _CalculateServerProcessOptions(); + +}; + +#endif // PROCESS_COORDINATOR_FACTORY_H \ No newline at end of file diff --git a/src/apps/haikudepot/server/ProcessNode.cpp b/src/apps/haikudepot/server/ProcessNode.cpp new file mode 100644 index 0000000000..259fd726a5 --- /dev/null +++ b/src/apps/haikudepot/server/ProcessNode.cpp @@ -0,0 +1,195 @@ +/* + * Copyright 2018, Andrew Lindesay . + * All rights reserved. Distributed under the terms of the MIT License. + */ + + +#include "ProcessNode.h" + +#include + +#include "AbstractProcess.h" +#include "Logger.h" + + +#define SPIN_UNTIL_STARTED_DELAY_MI 250 * 1000 + // quarter of a second + +#define TIMEOUT_UNTIL_STARTED_SECS 10 +#define TIMEOUT_UNTIL_STOPPED_SECS 10 + + +ProcessNode::ProcessNode(AbstractProcess* process) + : + fWorker(B_BAD_THREAD_ID), + fProcess(process) +{ +} + + +ProcessNode::~ProcessNode() +{ + if (fProcess != NULL) + delete fProcess; +} + + +AbstractProcess* +ProcessNode::Process() const +{ + return fProcess; +} + + +/*! This method will spin-lock the thread until the process is in one of the + states defined by the mask. + */ + +status_t +ProcessNode::_SpinUntilProcessState( + uint32 desiredStatesMask, uint32 timeoutSeconds) +{ + uint32 start = real_time_clock(); + + while (true) { + if ((Process()->ProcessState() & desiredStatesMask) != 0) + return B_OK; + + usleep(SPIN_UNTIL_STARTED_DELAY_MI); + + if (real_time_clock() - start > timeoutSeconds) { + printf("[Node<%s>] timeout waiting for process state\n", + Process()->Name()); + return B_ERROR; + } + } +} + + +/*! Considered to be protected from concurrent access by the ProcessCoordinator +*/ + +status_t +ProcessNode::StartProcess() +{ + if (fWorker != B_BAD_THREAD_ID) + return B_BUSY; + + if (Logger::IsInfoEnabled()) + printf("[Node<%s>] initiating\n", Process()->Name()); + + fWorker = spawn_thread(&_StartProcess, Process()->Name(), + B_NORMAL_PRIORITY, Process()); + + if (fWorker >= 0) { + resume_thread(fWorker); + return _SpinUntilProcessState(PROCESS_RUNNING | PROCESS_COMPLETE, + TIMEOUT_UNTIL_STARTED_SECS); + } + + return B_ERROR; +} + + +/*! Considered to be protected from concurrent access by the ProcessCoordinator +*/ + +status_t +ProcessNode::StopProcess() +{ + Process()->SetListener(NULL); + status_t stopResult = Process()->Stop(); + status_t waitResult = _SpinUntilProcessState(PROCESS_COMPLETE, + TIMEOUT_UNTIL_STOPPED_SECS); + + // if the thread is still running then it will be necessary to tear it + // down. + + if (waitResult != B_OK) { + printf("[%s] process did not stop within timeout - will be stopped " + "uncleanly", Process()->Name()); + kill_thread(fWorker); + } + + if (stopResult != B_OK) + return stopResult; + + if (waitResult != B_OK) + return waitResult; + + return B_OK; +} + + +/*! This method is the initial function that is invoked on starting a new + thread. It will start a process that is part of the bulk-load. + */ + +/*static*/ status_t +ProcessNode::_StartProcess(void* cookie) +{ + AbstractProcess* process = static_cast(cookie); + + if (Logger::IsInfoEnabled()) { + printf("[Node<%s>] starting process\n", process->Name()); + } + + process->Run(); + return B_OK; +} + + +void +ProcessNode::AddPredecessor(ProcessNode *node) +{ + fPredecessorNodes.Add(node); + node->_AddSuccessor(this); +} + + +int32 +ProcessNode::CountPredecessors() const +{ + return fPredecessorNodes.CountItems(); +} + + +ProcessNode* +ProcessNode::PredecessorAt(int32 index) const +{ + return fPredecessorNodes.ItemAt(index); +} + + +bool +ProcessNode::AllPredecessorsComplete() const +{ + for (int32 i = 0; i < CountPredecessors(); i++) { + if (PredecessorAt(i)->Process()->ProcessState() != PROCESS_COMPLETE) + return false; + } + + return true; +} + + +void +ProcessNode::_AddSuccessor(ProcessNode* node) +{ + fSuccessorNodes.Add(node); +} + + +int32 +ProcessNode::CountSuccessors() const +{ + return fSuccessorNodes.CountItems(); +} + + +ProcessNode* +ProcessNode::SuccessorAt(int32 index) const +{ + return fSuccessorNodes.ItemAt(index); +} + diff --git a/src/apps/haikudepot/server/ProcessNode.h b/src/apps/haikudepot/server/ProcessNode.h new file mode 100644 index 0000000000..f03c823745 --- /dev/null +++ b/src/apps/haikudepot/server/ProcessNode.h @@ -0,0 +1,58 @@ +/* + * Copyright 2018, Andrew Lindesay . + * All rights reserved. Distributed under the terms of the MIT License. + */ + + +#ifndef PROCESS_NODE_H +#define PROCESS_NODE_H + + +#include + +#include "List.h" + + +class AbstractProcess; + + +/*! This class is designed to be used by the ProcessCoordinator class. The + purpose of the class is to hold a process and also any dependent processes + of this one. This effectively creates a dependency tree of processes. This + class is also able to start and stop threads that run the process. +*/ + +class ProcessNode { +public: + ProcessNode(AbstractProcess* process); + virtual ~ProcessNode(); + + AbstractProcess* Process() const; + status_t StartProcess(); + status_t StopProcess(); + + void AddPredecessor(ProcessNode* node); + int32 CountPredecessors() const; + ProcessNode* PredecessorAt(int32 index) const; + bool AllPredecessorsComplete() const; + + int32 CountSuccessors() const; + ProcessNode* SuccessorAt(int32 index) const; + +private: + static status_t _StartProcess(void* cookie); + status_t _SpinUntilProcessState( + uint32 desiredStatesMask, + uint32 timeoutSeconds); + void _AddSuccessor(ProcessNode* node); + + thread_id fWorker; + AbstractProcess* fProcess; + List + fPredecessorNodes; + List + fSuccessorNodes; +}; + + +#endif // PROCESS_TREE_NODE_H diff --git a/src/apps/haikudepot/server/ServerHelper.cpp b/src/apps/haikudepot/server/ServerHelper.cpp index 69e00a6102..bf453af6c4 100644 --- a/src/apps/haikudepot/server/ServerHelper.cpp +++ b/src/apps/haikudepot/server/ServerHelper.cpp @@ -3,6 +3,7 @@ * All rights reserved. Distributed under the terms of the MIT License. */ + #include "ServerHelper.h" #include @@ -21,6 +22,7 @@ #undef B_TRANSLATION_CONTEXT #define B_TRANSLATION_CONTEXT "ServerHelper" + #define KEY_MSG_MINIMUM_VERSION "minimumVersion" #define KEY_HEADER_MINIMUM_VERSION "X-Desktop-Application-Minimum-Version" @@ -31,7 +33,7 @@ to the looper and then onto the user to see. */ -void +/*static*/ void ServerHelper::NotifyServerJsonRpcError(BMessage& error) { BMessage message(MSG_SERVER_ERROR); @@ -40,7 +42,7 @@ ServerHelper::NotifyServerJsonRpcError(BMessage& error) } -void +/*static*/ void ServerHelper::AlertServerJsonRpcError(BMessage* message) { BMessage error; @@ -84,7 +86,7 @@ ServerHelper::AlertServerJsonRpcError(BMessage* message) } -void +/*static*/ void ServerHelper::NotifyTransportError(status_t error) { switch (error) { @@ -105,7 +107,7 @@ ServerHelper::NotifyTransportError(status_t error) } -void +/*static*/ void ServerHelper::AlertTransportError(BMessage* message) { status_t errno = B_OK; @@ -139,7 +141,7 @@ ServerHelper::AlertTransportError(BMessage* message) } -void +/*static*/ void ServerHelper::NotifyClientTooOld(const BHttpHeaders& responseHeaders) { if (!ServerSettings::IsClientTooOld()) { @@ -157,7 +159,7 @@ ServerHelper::NotifyClientTooOld(const BHttpHeaders& responseHeaders) } -void +/*static*/ void ServerHelper::AlertClientTooOld(BMessage* message) { BString minimumVersion; @@ -182,14 +184,14 @@ ServerHelper::AlertClientTooOld(BMessage* message) } -bool +/*static*/ bool ServerHelper::IsNetworkAvailable() { return !ServerSettings::ForceNoNetwork() && IsPlatformNetworkAvailable(); } -bool +/*static*/ bool ServerHelper::IsPlatformNetworkAvailable() { BNetworkRoster& roster = BNetworkRoster::Default(); @@ -204,4 +206,4 @@ ServerHelper::IsPlatformNetworkAvailable() } return false; -} +} \ No newline at end of file diff --git a/src/apps/haikudepot/server/ServerHelper.h b/src/apps/haikudepot/server/ServerHelper.h index 01ad48b217..f7554ac01a 100644 --- a/src/apps/haikudepot/server/ServerHelper.h +++ b/src/apps/haikudepot/server/ServerHelper.h @@ -2,7 +2,6 @@ * Copyright 2017-2018, Andrew Lindesay . * All rights reserved. Distributed under the terms of the MIT License. */ - #ifndef SERVER_HELPER_H #define SERVER_HELPER_H @@ -14,20 +13,20 @@ class BMessage; class ServerHelper { public: - static bool IsNetworkAvailable(); - static bool IsPlatformNetworkAvailable(); + static bool IsNetworkAvailable(); + static bool IsPlatformNetworkAvailable(); - static void NotifyClientTooOld( + static void NotifyClientTooOld( const BHttpHeaders& responseHeaders ); - static void AlertClientTooOld(BMessage* message); + static void AlertClientTooOld(BMessage* message); - static void NotifyTransportError(status_t error); - static void AlertTransportError(BMessage* message); + static void NotifyTransportError(status_t error); + static void AlertTransportError(BMessage* message); - static void NotifyServerJsonRpcError( + static void NotifyServerJsonRpcError( BMessage& error); - static void AlertServerJsonRpcError( + static void AlertServerJsonRpcError( BMessage* message); }; diff --git a/src/apps/haikudepot/server/ServerIconExportUpdateProcess.cpp b/src/apps/haikudepot/server/ServerIconExportUpdateProcess.cpp index 521e355d20..1d1878ade0 100644 --- a/src/apps/haikudepot/server/ServerIconExportUpdateProcess.cpp +++ b/src/apps/haikudepot/server/ServerIconExportUpdateProcess.cpp @@ -3,75 +3,95 @@ * All rights reserved. Distributed under the terms of the MIT License. */ + #include "ServerIconExportUpdateProcess.h" #include #include #include -#include +#include +#include #include #include #include #include "HaikuDepotConstants.h" #include "Logger.h" +#include "ServerHelper.h" #include "ServerSettings.h" #include "StorageUtils.h" #include "TarArchiveService.h" +#undef B_TRANSLATION_CONTEXT +#define B_TRANSLATION_CONTEXT "ServerIconExportUpdateProcess" + + /*! This constructor will locate the cached data in a standardized location */ ServerIconExportUpdateProcess::ServerIconExportUpdateProcess( - AbstractServerProcessListener* listener, - const BPath& localStorageDirectoryPath, Model* model, - uint32 options) + uint32 serverProcessOptions) : - AbstractServerProcess(listener, options), - fLocalStorageDirectoryPath(localStorageDirectoryPath), + AbstractServerProcess(serverProcessOptions), fModel(model), - fLocalIconStore(LocalIconStore(localStorageDirectoryPath)), fCountIconsSet(0) { + if (fModel->IconStoragePath(fLocalIconStoragePath) != B_OK) { + printf("[%s] unable to obtain the path for storing icons\n", Name()); + fLocalIconStoragePath.Unset(); + fLocalIconStore = NULL; + } else { + fLocalIconStore = new LocalIconStore(fLocalIconStoragePath); + } } ServerIconExportUpdateProcess::~ServerIconExportUpdateProcess() { + delete fLocalIconStore; } const char* -ServerIconExportUpdateProcess::Name() +ServerIconExportUpdateProcess::Name() const { return "ServerIconExportUpdateProcess"; } +const char* +ServerIconExportUpdateProcess::Description() const +{ + return B_TRANSLATE("Synchronizing icons"); +} + + status_t ServerIconExportUpdateProcess::RunInternal() { status_t result = B_OK; + if (NULL == fLocalIconStore || fLocalIconStoragePath.Path() == NULL) + result = B_ERROR; + if (IsSuccess(result) && HasOption(SERVER_PROCESS_DROP_CACHE)) { - result = StorageUtils::RemoveDirectoryContents( - fLocalStorageDirectoryPath); + result = StorageUtils::RemoveDirectoryContents(fLocalIconStoragePath); } - if (IsSuccess(result)) { + if (result == B_OK) { bool hasData; result = HasLocalData(&hasData); - if (IsSuccess(result) && ShouldAttemptNetworkDownload(hasData)) - result = DownloadAndUnpack(); + if (result == B_OK && ShouldAttemptNetworkDownload(hasData)) + result = _DownloadAndUnpack(); if (IsSuccess(result)) { status_t hasDataResult = HasLocalData(&hasData); - if (IsSuccess(hasDataResult) && !hasData) + if (hasDataResult == B_OK && !hasData) result = HD_ERR_NO_DATA; } } @@ -83,23 +103,81 @@ ServerIconExportUpdateProcess::RunInternal() } +status_t +ServerIconExportUpdateProcess::Populate() +{ + BStopWatch watch("ServerIconExportUpdateProcess::Populate", true); + DepotList depots = fModel->Depots(); + status_t result = B_OK; + + { + AutoLocker locker(fModel->Lock()); + depots = fModel->Depots(); + } + + if (Logger::IsDebugEnabled()) { + printf("[%s] will populate icons for %" B_PRId32 " depots\n", Name(), + depots.CountItems()); + } + + for (int32 i = 0; + (i < depots.CountItems()) && !WasStopped() && (result == B_OK); + i++) { + AutoLocker locker(fModel->Lock()); + DepotInfo depotInfo = depots.ItemAtFast(i); + result = PopulateForDepot(depotInfo); + } + + if (Logger::IsInfoEnabled()) { + double secs = watch.ElapsedTime() / 1000000.0; + printf("[%s] did populate %" B_PRId32 " packages' icons (%6.3g secs)\n", + Name(), fCountIconsSet, secs); + } + + return result; +} + + +/*! This method assumes that the model lock has been acquired */ + +status_t +ServerIconExportUpdateProcess::PopulateForDepot(const DepotInfo& depot) +{ + printf("[%s] will populate icons for depot [%s]\n", + Name(), depot.Name().String()); + status_t result = B_OK; + PackageList packages = depot.Packages(); + for(int32 j = 0; + (j < packages.CountItems()) && !WasStopped() && (result == B_OK); + j++) { + const PackageInfoRef& packageInfoRef = packages.ItemAtFast(j); + result = PopulateForPkg(packageInfoRef); + + if (result == B_FILE_NOT_FOUND) + result = B_OK; + } + + return result; +} + + +/*! This method assumes that the model lock has been acquired */ + status_t ServerIconExportUpdateProcess::PopulateForPkg(const PackageInfoRef& package) { BPath bestIconPath; - if ( fLocalIconStore.TryFindIconPath( + if ( fLocalIconStore->TryFindIconPath( package->Name(), bestIconPath) == B_OK) { BFile bestIconFile(bestIconPath.Path(), O_RDONLY); BitmapRef bitmapRef(new(std::nothrow)SharedBitmap(bestIconFile), true); - // TODO; somehow handle the locking! - //BAutolock locker(&fLock); package->SetIcon(bitmapRef); if (Logger::IsDebugEnabled()) { - fprintf(stdout, "have set the package icon for [%s] from [%s]\n", - package->Name().String(), bestIconPath.Path()); + printf("[%s] have set the package icon for [%s] from [%s]\n", + Name(), package->Name().String(), bestIconPath.Path()); } fCountIconsSet++; @@ -108,105 +186,141 @@ ServerIconExportUpdateProcess::PopulateForPkg(const PackageInfoRef& package) } if (Logger::IsDebugEnabled()) { - fprintf(stdout, "did not set the package icon for [%s]; no data\n", - package->Name().String()); + printf("[%s] did not set the package icon for [%s]; no data\n", + Name(), package->Name().String()); } return B_FILE_NOT_FOUND; } -bool -ServerIconExportUpdateProcess::ConsumePackage( - const PackageInfoRef& packageInfoRef, void* context) -{ - PopulateForPkg(packageInfoRef); - return !WasStopped(); -} - - status_t -ServerIconExportUpdateProcess::Populate() -{ - BStopWatch watch("ServerIconExportUpdateProcess::Populate", true); - fModel->ForAllPackages(this, NULL); - - if (Logger::IsInfoEnabled()) { - double secs = watch.ElapsedTime() / 1000000.0; - fprintf(stdout, "did populate %" B_PRId32 " packages' icons" - " (%6.3g secs)\n", fCountIconsSet, secs); - } - - return B_OK; -} - - -status_t -ServerIconExportUpdateProcess::DownloadAndUnpack() +ServerIconExportUpdateProcess::_DownloadAndUnpack() { BPath tarGzFilePath(tmpnam(NULL)); status_t result = B_OK; - printf("will start fetching icons\n"); + printf("[%s] will start fetching icons\n", Name()); - result = Download(tarGzFilePath); + result = _Download(tarGzFilePath); + + switch (result) { + case HD_ERR_NOT_MODIFIED: + printf("[%s] icons not modified - will use existing\n", Name()); + return result; + break; + case B_OK: + return _Unpack(tarGzFilePath); + default: + return (_HandleDownloadFailure() != B_OK) ? result : B_OK; + } +} + + +/*! if the download failed, but there are existing icons in place to use + then use those icons. To detect the existing files, look for the + icons' meta-info file. +*/ + +status_t +ServerIconExportUpdateProcess::_HandleDownloadFailure() +{ + bool hasData; + status_t result = HasLocalData(&hasData); if (result == B_OK) { - printf("delete any existing stored data\n"); - StorageUtils::RemoveDirectoryContents(fLocalStorageDirectoryPath); - - BFile *tarGzFile = new BFile(tarGzFilePath.Path(), O_RDONLY); - BDataIO* tarIn; - - BZlibDecompressionParameters* zlibDecompressionParameters - = new BZlibDecompressionParameters(); - - result = BZlibCompressionAlgorithm() - .CreateDecompressingInputStream(tarGzFile, - zlibDecompressionParameters, tarIn); - - if (result == B_OK) { - BStopWatch watch("ServerIconExportUpdateProcess::DownloadAndUnpack_Unpack", true); - - result = TarArchiveService::Unpack(*tarIn, - fLocalStorageDirectoryPath, NULL); - - if (result == B_OK) { - double secs = watch.ElapsedTime() / 1000000.0; - fprintf(stdout, "did unpack icon tgz in (%6.3g secs)\n", secs); - - if (0 != remove(tarGzFilePath.Path())) { - fprintf(stdout, "unable to delete the temporary tgz path; " - "%s\n", tarGzFilePath.Path()); - } - } + if (hasData) { + printf("[%s] failed to update data, but have old data anyway " + "so will carry on with that\n", Name()); + } else { + printf("[%s] failed to obtain data\n", Name()); + result = HD_ERR_NO_DATA; } - - delete tarGzFile; - - printf("did complete fetching icons\n"); + } else { + printf("[%s] unable to detect if there is local data\n", Name()); } return result; } +/*! The tar-ball data of icons has arrived and so old data needs to be purged + to make way for the new data and the new data needs to be unpacked. +*/ + +status_t +ServerIconExportUpdateProcess::_Unpack(BPath& tarGzFilePath) +{ + status_t result; + printf("[%s] delete any existing stored data\n", Name()); + StorageUtils::RemoveDirectoryContents(fLocalIconStoragePath); + + BFile *tarGzFile = new BFile(tarGzFilePath.Path(), O_RDONLY); + BDataIO* tarIn; + + BZlibDecompressionParameters* zlibDecompressionParameters + = new BZlibDecompressionParameters(); + + result = BZlibCompressionAlgorithm() + .CreateDecompressingInputStream(tarGzFile, + zlibDecompressionParameters, tarIn); + + if (result == B_OK) { + BStopWatch watch( + "ServerIconExportUpdateProcess::DownloadAndUnpack_Unpack", + true); + + result = TarArchiveService::Unpack(*tarIn, + fLocalIconStoragePath, NULL); + + if (result == B_OK) { + double secs = watch.ElapsedTime() / 1000000.0; + printf("[%s] did unpack icon tgz in (%6.3g secs)\n", Name(), + secs); + + if (0 != remove(tarGzFilePath.Path())) { + printf("unable to delete the temporary tgz path; %s\n", + tarGzFilePath.Path()); + } + } + } + + delete tarGzFile; + printf("[%s] did complete unpacking icons\n", Name()); + return result; +} + + status_t ServerIconExportUpdateProcess::HasLocalData(bool* result) const { BPath path; + status_t status = GetStandardMetaDataPath(path); + + if (status != B_OK) + return status; + off_t size; - GetStandardMetaDataPath(path); - return StorageUtils::ExistsObject(path, result, NULL, &size) - && size > 0; + + status = StorageUtils::ExistsObject(path, result, NULL, &size); + + if (status == B_OK && size == 0) + *result = false; + + return status; } -void +status_t ServerIconExportUpdateProcess::GetStandardMetaDataPath(BPath& path) const { - path.SetTo(fLocalStorageDirectoryPath.Path()); + status_t result = fModel->IconStoragePath(path); + + if (result != B_OK) + return result; + path.Append("hicn/info.json"); + return B_OK; } @@ -220,7 +334,7 @@ ServerIconExportUpdateProcess::GetStandardMetaDataJsonPath( status_t -ServerIconExportUpdateProcess::Download(BPath& tarGzFilePath) +ServerIconExportUpdateProcess::_Download(BPath& tarGzFilePath) { return DownloadToLocalFileAtomically(tarGzFilePath, ServerSettings::CreateFullUrl("/__pkgicon/all.tar.gz")); diff --git a/src/apps/haikudepot/server/ServerIconExportUpdateProcess.h b/src/apps/haikudepot/server/ServerIconExportUpdateProcess.h index ceb9c83058..2df940cd61 100644 --- a/src/apps/haikudepot/server/ServerIconExportUpdateProcess.h +++ b/src/apps/haikudepot/server/ServerIconExportUpdateProcess.h @@ -1,52 +1,53 @@ /* - * Copyright 2017, Andrew Lindesay . + * Copyright 2017-2018, Andrew Lindesay . * All rights reserved. Distributed under the terms of the MIT License. */ - #ifndef SERVER_ICON_EXPORT_UPDATE_PROCESS_H #define SERVER_ICON_EXPORT_UPDATE_PROCESS_H -#include "AbstractServerProcess.h" -#include "LocalIconStore.h" -#include "Model.h" - #include #include #include #include +#include "AbstractServerProcess.h" +#include "LocalIconStore.h" +#include "Model.h" -class ServerIconExportUpdateProcess : - public AbstractServerProcess, public PackageConsumer { + +class DumpExportPkg; + + +class ServerIconExportUpdateProcess : public AbstractServerProcess { public: ServerIconExportUpdateProcess( - AbstractServerProcessListener* listener, - const BPath& localStorageDirectoryPath, - Model* model, uint32 options); + Model* model, uint32 serverProcessOptions); virtual ~ServerIconExportUpdateProcess(); - const char* Name(); + const char* Name() const; + const char* Description() const; + status_t RunInternal(); - virtual bool ConsumePackage( - const PackageInfoRef& packageInfoRef, - void *context); protected: status_t PopulateForPkg(const PackageInfoRef& package); + status_t PopulateForDepot(const DepotInfo& depot); status_t Populate(); - status_t DownloadAndUnpack(); status_t HasLocalData(bool* result) const; - void GetStandardMetaDataPath(BPath& path) const; + status_t GetStandardMetaDataPath(BPath& path) const; void GetStandardMetaDataJsonPath( BString& jsonPath) const; private: - status_t Download(BPath& tarGzFilePath); + status_t _Unpack(BPath& tarGzFilePath); + status_t _HandleDownloadFailure(); + status_t _DownloadAndUnpack(); + status_t _Download(BPath& tarGzFilePath); - BPath fLocalStorageDirectoryPath; Model* fModel; - LocalIconStore fLocalIconStore; + BPath fLocalIconStoragePath; + LocalIconStore* fLocalIconStore; int32 fCountIconsSet; }; diff --git a/src/apps/haikudepot/server/PkgDataUpdateProcess.cpp b/src/apps/haikudepot/server/ServerPkgDataUpdateProcess.cpp similarity index 63% rename from src/apps/haikudepot/server/PkgDataUpdateProcess.cpp rename to src/apps/haikudepot/server/ServerPkgDataUpdateProcess.cpp index 45361c7dc1..b87f7450f9 100644 --- a/src/apps/haikudepot/server/PkgDataUpdateProcess.cpp +++ b/src/apps/haikudepot/server/ServerPkgDataUpdateProcess.cpp @@ -3,13 +3,16 @@ * All rights reserved. Distributed under the terms of the MIT License. */ -#include "PkgDataUpdateProcess.h" + +#include "ServerPkgDataUpdateProcess.h" #include #include #include -#include +#include +#include +#include #include #include #include @@ -25,19 +28,22 @@ #include "HaikuDepotConstants.h" +#undef B_TRANSLATION_CONTEXT +#define B_TRANSLATION_CONTEXT "ServerPkgDataUpdateProcess" + + /*! This package listener (not at the JSON level) is feeding in the packages as they are parsed and processing them. */ -class PackageFillingPkgListener : - public DumpExportPkgListener, public PackageConsumer { +class PackageFillingPkgListener : public DumpExportPkgListener { public: PackageFillingPkgListener(Model *model, BString& depotName, Stoppable* stoppable); virtual ~PackageFillingPkgListener(); virtual bool ConsumePackage(const PackageInfoRef& package, - void *context); + DumpExportPkg* pkg); virtual bool Handle(DumpExportPkg* item); virtual void Complete(); @@ -99,9 +105,8 @@ PackageFillingPkgListener::IndexOfCategoryByName( bool PackageFillingPkgListener::ConsumePackage(const PackageInfoRef& package, - void *context) + DumpExportPkg* pkg) { - DumpExportPkg* pkg = static_cast(context); int32 i; // Collects all of the changes here into one set of notifications to @@ -195,7 +200,28 @@ PackageFillingPkgListener::Count() bool PackageFillingPkgListener::Handle(DumpExportPkg* pkg) { - fModel->ForPackageByNameInDepot(fDepotName, *(pkg->Name()), this, pkg); + const DepotInfo* depotInfo = fModel->DepotForName(fDepotName); + + if (depotInfo != NULL) { + BString packageName = *(pkg->Name()); + int32 packageIndex = depotInfo->PackageIndexByName(packageName); + + if (-1 != packageIndex) { + PackageList packages = depotInfo->Packages(); + const PackageInfoRef& packageInfoRef = + packages.ItemAtFast(packageIndex); + + AutoLocker locker(fModel->Lock()); + ConsumePackage(packageInfoRef, pkg); + } else { + printf("[PackageFillingPkgListener] unable to find the pkg [%s]\n", + packageName.String()); + } + } else { + printf("[PackageFillingPkgListener] unable to find the depot [%s]\n", + fDepotName.String()); + } + return !fStoppable->WasStopped(); } @@ -206,92 +232,143 @@ PackageFillingPkgListener::Complete() } -PkgDataUpdateProcess::PkgDataUpdateProcess( - AbstractServerProcessListener* listener, - const BPath& localFilePath, +ServerPkgDataUpdateProcess::ServerPkgDataUpdateProcess( BString naturalLanguageCode, - BString repositorySourceCode, BString depotName, Model *model, - uint32 options) + uint32 serverProcessOptions) : - AbstractSingleFileServerProcess(listener, options), - fLocalFilePath(localFilePath), + AbstractSingleFileServerProcess(serverProcessOptions), fNaturalLanguageCode(naturalLanguageCode), - fRepositorySourceCode(repositorySourceCode), fModel(model), fDepotName(depotName) { - fName.SetToFormat("PkgDataUpdateProcess<%s>", depotName.String()); + fName.SetToFormat("ServerPkgDataUpdateProcess<%s>", depotName.String()); + fDescription.SetTo( + B_TRANSLATE("Synchronizing package data for repository " + "'%REPO_NAME%'")); + fDescription.ReplaceAll("%REPO_NAME%", depotName.String()); } -PkgDataUpdateProcess::~PkgDataUpdateProcess() +ServerPkgDataUpdateProcess::~ServerPkgDataUpdateProcess() { } const char* -PkgDataUpdateProcess::Name() +ServerPkgDataUpdateProcess::Name() const { return fName.String(); } +const char* +ServerPkgDataUpdateProcess::Description() const +{ + return fDescription.String(); +} + + BString -PkgDataUpdateProcess::UrlPathComponent() +ServerPkgDataUpdateProcess::UrlPathComponent() { BString urlPath; urlPath.SetToFormat("/__pkg/all-%s-%s.json.gz", - fRepositorySourceCode.String(), + _DeriveWebAppRepositorySourceCode().String(), fNaturalLanguageCode.String()); return urlPath; } -BPath& -PkgDataUpdateProcess::LocalPath() +status_t +ServerPkgDataUpdateProcess::GetLocalPath(BPath& path) const { - return fLocalFilePath; + BString webAppRepositorySourceCode = _DeriveWebAppRepositorySourceCode(); + + if (!webAppRepositorySourceCode.IsEmpty()) { + return fModel->DumpExportPkgDataPath(path, webAppRepositorySourceCode); + } + + return B_ERROR; } status_t -PkgDataUpdateProcess::ProcessLocalData() +ServerPkgDataUpdateProcess::ProcessLocalData() { - BStopWatch watch("PkgDataUpdateProcess::ProcessLocalData", true); + BStopWatch watch("ServerPkgDataUpdateProcess::ProcessLocalData", true); PackageFillingPkgListener* itemListener = new PackageFillingPkgListener(fModel, fDepotName, this); + ObjectDeleter + itemListenerDeleter(itemListener); BulkContainerDumpExportPkgJsonListener* listener = new BulkContainerDumpExportPkgJsonListener(itemListener); + ObjectDeleter + listenerDeleter(listener); - status_t result = ParseJsonFromFileWithListener(listener, fLocalFilePath); + BPath localPath; + status_t result = GetLocalPath(localPath); - if (Logger::IsInfoEnabled()) { - double secs = watch.ElapsedTime() / 1000000.0; - fprintf(stdout, "[%s] did process %" B_PRIi32 " packages' data " - "in (%6.3g secs)\n", Name(), itemListener->Count(), secs); - } + if (result != B_OK) + return result; + + result = ParseJsonFromFileWithListener(listener, localPath); if (B_OK != result) return result; + if (Logger::IsInfoEnabled()) { + double secs = watch.ElapsedTime() / 1000000.0; + printf("[%s] did process %" B_PRIi32 " packages' data " + "in (%6.3g secs)\n", Name(), itemListener->Count(), secs); + } + return listener->ErrorStatus(); } -void -PkgDataUpdateProcess::GetStandardMetaDataPath(BPath& path) const +status_t +ServerPkgDataUpdateProcess::GetStandardMetaDataPath(BPath& path) const { - path.SetTo(fLocalFilePath.Path()); + return GetLocalPath(path); } void -PkgDataUpdateProcess::GetStandardMetaDataJsonPath( +ServerPkgDataUpdateProcess::GetStandardMetaDataJsonPath( BString& jsonPath) const { jsonPath.SetTo("$.info"); } + + +BString +ServerPkgDataUpdateProcess::_DeriveWebAppRepositorySourceCode() const +{ + const DepotInfo* depot = fModel->DepotForName(fDepotName); + + if (depot == NULL) { + return BString(); + } + + return depot->WebAppRepositorySourceCode(); +} + + +status_t +ServerPkgDataUpdateProcess::RunInternal() +{ + if (_DeriveWebAppRepositorySourceCode().IsEmpty()) { + if (Logger::IsInfoEnabled()) { + printf("[%s] am not updating data for depot [%s] as there is no" + " web app repository source code available\n", + Name(), fDepotName.String()); + } + return B_OK; + } + + return AbstractSingleFileServerProcess::RunInternal(); +} diff --git a/src/apps/haikudepot/server/PkgDataUpdateProcess.h b/src/apps/haikudepot/server/ServerPkgDataUpdateProcess.h similarity index 57% rename from src/apps/haikudepot/server/PkgDataUpdateProcess.h rename to src/apps/haikudepot/server/ServerPkgDataUpdateProcess.h index 7d1486a40e..7ba0b3d323 100644 --- a/src/apps/haikudepot/server/PkgDataUpdateProcess.h +++ b/src/apps/haikudepot/server/ServerPkgDataUpdateProcess.h @@ -1,54 +1,53 @@ /* - * Copyright 2017, Andrew Lindesay . + * Copyright 2017-2018, Andrew Lindesay . * All rights reserved. Distributed under the terms of the MIT License. */ - #ifndef PACKAGE_DATA_UPDATE_PROCESS_H #define PACKAGE_DATA_UPDATE_PROCESS_H #include "AbstractSingleFileServerProcess.h" -#include "Model.h" -#include "PackageInfo.h" - #include #include #include #include +#include "Model.h" +#include "PackageInfo.h" -class PkgDataUpdateProcess : public AbstractSingleFileServerProcess { + +class ServerPkgDataUpdateProcess : public AbstractSingleFileServerProcess { public: - PkgDataUpdateProcess( - AbstractServerProcessListener* listener, - const BPath& localFilePath, + ServerPkgDataUpdateProcess( BString naturalLanguageCode, - BString repositorySourceCode, BString depotName, Model *model, - uint32 options); - virtual ~PkgDataUpdateProcess(); + uint32 serverProcessOptions); + virtual ~ServerPkgDataUpdateProcess(); - const char* Name(); + const char* Name() const; + const char* Description() const; protected: - void GetStandardMetaDataPath(BPath& path) const; + virtual status_t RunInternal(); + + status_t GetStandardMetaDataPath(BPath& path) const; void GetStandardMetaDataJsonPath( BString& jsonPath) const; BString UrlPathComponent(); status_t ProcessLocalData(); - BPath& LocalPath(); + status_t GetLocalPath(BPath& path) const; private: + BString _DeriveWebAppRepositorySourceCode() const; - BPath fLocalFilePath; BString fNaturalLanguageCode; - BString fRepositorySourceCode; Model* fModel; BString fDepotName; BString fName; + BString fDescription; }; diff --git a/src/apps/haikudepot/server/RepositoryDataUpdateProcess.cpp b/src/apps/haikudepot/server/ServerRepositoryDataUpdateProcess.cpp similarity index 65% rename from src/apps/haikudepot/server/RepositoryDataUpdateProcess.cpp rename to src/apps/haikudepot/server/ServerRepositoryDataUpdateProcess.cpp index 44e7002dec..cc4b78e109 100644 --- a/src/apps/haikudepot/server/RepositoryDataUpdateProcess.cpp +++ b/src/apps/haikudepot/server/ServerRepositoryDataUpdateProcess.cpp @@ -3,12 +3,15 @@ * All rights reserved. Distributed under the terms of the MIT License. */ -#include "RepositoryDataUpdateProcess.h" + +#include "ServerRepositoryDataUpdateProcess.h" #include #include #include +#include +#include #include #include @@ -20,6 +23,10 @@ #include "DumpExportRepositoryJsonListener.h" +#undef B_TRANSLATION_CONTEXT +#define B_TRANSLATION_CONTEXT "ServerRepositoryDataUpdateProcess" + + /*! This repository listener (not at the JSON level) is feeding in the repositories as they are parsed and processing them. Processing includes finding the matching depot record and coupling the data @@ -91,14 +98,17 @@ DepotMatchingRepositoryListener::MapDepot(const DepotInfo& depot, void *context) BString(*repositorySourceCode)); if (Logger::IsDebugEnabled()) { - printf("associated dept [%s] (%s) with server repository " - "source [%s] (%s)\n", modifiedDepotInfo.Name().String(), - modifiedDepotInfo.BaseURL().String(), + printf("[DepotMatchingRepositoryListener] associated depot [%s] (%s) " + "with server repository source [%s] (%s)\n", + modifiedDepotInfo.Name().String(), + modifiedDepotInfo.URL().String(), repositorySourceCode->String(), repositoryAndRepositorySource->repositorySource->Url()->String()); } else { - printf("associated depot [%s] with server repository source [%s]\n", - modifiedDepotInfo.Name().String(), repositorySourceCode->String()); + printf("[DepotMatchingRepositoryListener] associated depot [%s] with " + "server repository source [%s]\n", + modifiedDepotInfo.Name().String(), + repositorySourceCode->String()); } return modifiedDepotInfo; @@ -116,20 +126,11 @@ DepotMatchingRepositoryListener::Handle(DumpExportRepository* repository) repositoryAndRepositorySource.repositorySource = repository->RepositorySourcesItemAt(i); - BString* baseURL = repositoryAndRepositorySource - .repositorySource->Url(); - BString* URL = repositoryAndRepositorySource + BString* repoInfoURL = repositoryAndRepositorySource .repositorySource->RepoInfoUrl(); - // to be simplified soon because there will no longer be a need to - // check for the baseURL. - - if ((baseURL != NULL && !baseURL->IsEmpty()) - || (URL != NULL && !URL->IsEmpty())) { - fModel->ReplaceDepotByUrl( - URL == NULL ? BString() : *URL, - baseURL == NULL ? BString() : *baseURL, - this, + if (!repoInfoURL->IsEmpty()) { + fModel->ReplaceDepotByUrl(*repoInfoURL, this, &repositoryAndRepositorySource); } } @@ -144,72 +145,86 @@ DepotMatchingRepositoryListener::Complete() } -RepositoryDataUpdateProcess::RepositoryDataUpdateProcess( - AbstractServerProcessListener* listener, - const BPath& localFilePath, +ServerRepositoryDataUpdateProcess::ServerRepositoryDataUpdateProcess( Model* model, - uint32 options) + uint32 serverProcessOptions) : - AbstractSingleFileServerProcess(listener, options), - fLocalFilePath(localFilePath), + AbstractSingleFileServerProcess(serverProcessOptions), fModel(model) { } -RepositoryDataUpdateProcess::~RepositoryDataUpdateProcess() +ServerRepositoryDataUpdateProcess::~ServerRepositoryDataUpdateProcess() { } const char* -RepositoryDataUpdateProcess::Name() +ServerRepositoryDataUpdateProcess::Name() const { - return "RepositoryDataUpdateProcess"; + return "ServerRepositoryDataUpdateProcess"; +} + + +const char* +ServerRepositoryDataUpdateProcess::Description() const +{ + return B_TRANSLATE("Synchronizing meta-data about repositories"); } BString -RepositoryDataUpdateProcess::UrlPathComponent() +ServerRepositoryDataUpdateProcess::UrlPathComponent() { return BString("/__repository/all-en.json.gz"); } -BPath& -RepositoryDataUpdateProcess::LocalPath() +status_t +ServerRepositoryDataUpdateProcess::GetLocalPath(BPath& path) const { - return fLocalFilePath; + return fModel->DumpExportRepositoryDataPath(path); } status_t -RepositoryDataUpdateProcess::ProcessLocalData() +ServerRepositoryDataUpdateProcess::ProcessLocalData() { DepotMatchingRepositoryListener* itemListener = new DepotMatchingRepositoryListener(fModel, this); + ObjectDeleter + itemListenerDeleter(itemListener); BulkContainerDumpExportRepositoryJsonListener* listener = new BulkContainerDumpExportRepositoryJsonListener(itemListener); + ObjectDeleter + listenerDeleter(listener); - status_t result = ParseJsonFromFileWithListener(listener, fLocalFilePath); + BPath localPath; + status_t result = GetLocalPath(localPath); - if (B_OK != result) + if (result != B_OK) + return result; + + result = ParseJsonFromFileWithListener(listener, localPath); + + if (result != B_OK) return result; return listener->ErrorStatus(); } -void -RepositoryDataUpdateProcess::GetStandardMetaDataPath(BPath& path) const +status_t +ServerRepositoryDataUpdateProcess::GetStandardMetaDataPath(BPath& path) const { - path.SetTo(fLocalFilePath.Path()); + return GetLocalPath(path); } void -RepositoryDataUpdateProcess::GetStandardMetaDataJsonPath( +ServerRepositoryDataUpdateProcess::GetStandardMetaDataJsonPath( BString& jsonPath) const { jsonPath.SetTo("$.info"); diff --git a/src/apps/haikudepot/server/RepositoryDataUpdateProcess.h b/src/apps/haikudepot/server/ServerRepositoryDataUpdateProcess.h similarity index 50% rename from src/apps/haikudepot/server/RepositoryDataUpdateProcess.h rename to src/apps/haikudepot/server/ServerRepositoryDataUpdateProcess.h index 5bc48900c3..5be67d18fd 100644 --- a/src/apps/haikudepot/server/RepositoryDataUpdateProcess.h +++ b/src/apps/haikudepot/server/ServerRepositoryDataUpdateProcess.h @@ -1,45 +1,46 @@ /* - * Copyright 2017, Andrew Lindesay . + * Copyright 2017-2018, Andrew Lindesay . * All rights reserved. Distributed under the terms of the MIT License. */ - #ifndef REPOSITORY_DATA_UPDATE_PROCESS_H #define REPOSITORY_DATA_UPDATE_PROCESS_H #include "AbstractSingleFileServerProcess.h" -#include "Model.h" -#include "PackageInfo.h" - #include #include #include #include +#include "Model.h" +#include "PackageInfo.h" -class RepositoryDataUpdateProcess : public AbstractSingleFileServerProcess { + +#undef B_TRANSLATION_CONTEXT +#define B_TRANSLATION_CONTEXT "ServerRepositoryDataUpdateProcess" + + +class ServerRepositoryDataUpdateProcess : public AbstractSingleFileServerProcess { public: - RepositoryDataUpdateProcess( - AbstractServerProcessListener* listener, - const BPath& localFilePath, - Model* model, uint32 options); - virtual ~RepositoryDataUpdateProcess(); + ServerRepositoryDataUpdateProcess( + Model* model, uint32 serverProcessOptions); + virtual ~ServerRepositoryDataUpdateProcess(); - const char* Name(); + const char* Name() const; + const char* Description() const; protected: - void GetStandardMetaDataPath(BPath& path) const; + status_t GetStandardMetaDataPath(BPath& path) const; void GetStandardMetaDataJsonPath( BString& jsonPath) const; BString UrlPathComponent(); status_t ProcessLocalData(); - BPath& LocalPath(); + status_t GetLocalPath(BPath& path) const; private: - BPath fLocalFilePath; Model* fModel; }; diff --git a/src/apps/haikudepot/server/StandardMetaData.cpp b/src/apps/haikudepot/server/StandardMetaData.cpp index 5da9acb7d1..19505f4561 100644 --- a/src/apps/haikudepot/server/StandardMetaData.cpp +++ b/src/apps/haikudepot/server/StandardMetaData.cpp @@ -3,8 +3,10 @@ * All rights reserved. Distributed under the terms of the MIT License. */ + #include "StandardMetaData.h" + StandardMetaData::StandardMetaData() { fCreateTimestamp = 0; diff --git a/src/apps/haikudepot/server/StandardMetaDataJsonEventListener.cpp b/src/apps/haikudepot/server/StandardMetaDataJsonEventListener.cpp index dbdb5dda0d..e2feb83082 100644 --- a/src/apps/haikudepot/server/StandardMetaDataJsonEventListener.cpp +++ b/src/apps/haikudepot/server/StandardMetaDataJsonEventListener.cpp @@ -3,10 +3,12 @@ * All rights reserved. Distributed under the terms of the MIT License. */ + #include "StandardMetaDataJsonEventListener.h" #include "stdio.h" + #define KEY_CREATE_TIMESTAMP "createTimestamp" #define KEY_DATA_MODIFIED_TIMESTAMP "dataModifiedTimestamp" diff --git a/src/apps/haikudepot/server/StandardMetaDataJsonEventListener.h b/src/apps/haikudepot/server/StandardMetaDataJsonEventListener.h index 7d4be36d2f..0cc11c772c 100644 --- a/src/apps/haikudepot/server/StandardMetaDataJsonEventListener.h +++ b/src/apps/haikudepot/server/StandardMetaDataJsonEventListener.h @@ -11,8 +11,10 @@ #include #include + class SmdStackedEventListener; + class StandardMetaDataJsonEventListener : public BJsonEventListener { friend class SmdStackedEventListener; public: diff --git a/src/apps/haikudepot/tar/TarArchiveService.cpp b/src/apps/haikudepot/tar/TarArchiveService.cpp index c645003cfa..aa7e64b186 100644 --- a/src/apps/haikudepot/tar/TarArchiveService.cpp +++ b/src/apps/haikudepot/tar/TarArchiveService.cpp @@ -1,8 +1,9 @@ /* - * Copyright 2017, Andrew Lindesay . + * Copyright 2017-2018, Andrew Lindesay . * All rights reserved. Distributed under the terms of the MIT License. */ + #include "TarArchiveService.h" #include @@ -38,7 +39,8 @@ TarArchiveService::Unpack(BDataIO& tarDataIo, BPath& targetDirectory, count_items_read++; if (0 == memcmp(zero_buffer, buffer, sizeof zero_buffer)) { - fprintf(stdout, "detected end of tar-ball\n"); + if (Logger::IsDebugEnabled()) + printf("detected end of tar-ball\n"); return B_OK; // end of tar-ball. } else { TarArchiveHeader* header = TarArchiveHeader::CreateFromBlock( diff --git a/src/apps/haikudepot/tar/TarArchiveService.h b/src/apps/haikudepot/tar/TarArchiveService.h index b49e6da3b6..e6b3363a36 100644 --- a/src/apps/haikudepot/tar/TarArchiveService.h +++ b/src/apps/haikudepot/tar/TarArchiveService.h @@ -1,12 +1,10 @@ /* - * Copyright 2017, Andrew Lindesay . + * Copyright 2017-2018, Andrew Lindesay . * All rights reserved. Distributed under the terms of the MIT License. */ - #ifndef TAR_ARCHIVE_SERVICE_H #define TAR_ARCHIVE_SERVICE_H -#include "AbstractServerProcess.h" #include "Stoppable.h" #include "TarArchiveHeader.h" diff --git a/src/apps/haikudepot/ui/App.cpp b/src/apps/haikudepot/ui/App.cpp index 8a818402a4..0dc754b5e7 100644 --- a/src/apps/haikudepot/ui/App.cpp +++ b/src/apps/haikudepot/ui/App.cpp @@ -68,7 +68,7 @@ App::QuitRequested() _StoreSettings(windowSettings); } - return true; + return BApplication::QuitRequested(); } @@ -93,7 +93,7 @@ App::MessageReceived(BMessage* message) case MSG_MAIN_WINDOW_CLOSED: { BMessage windowSettings; - if (message->FindMessage("window settings", + if (message->FindMessage(KEY_WINDOW_SETTINGS, &windowSettings) == B_OK) { _StoreSettings(windowSettings); } @@ -116,6 +116,10 @@ App::MessageReceived(BMessage* message) ServerHelper::AlertServerJsonRpcError(message); break; + case MSG_ALERT_SIMPLE_ERROR: + _AlertSimpleError(message); + break; + case MSG_SERVER_DATA_CHANGED: fMainWindow->PostMessage(message); break; @@ -308,6 +312,29 @@ App::ArgvReceived(int32 argc, char* argv[]) } +/*! This method will display an alert based on a message. This message arrives + from a number of possible background threads / processes in the application. +*/ + +void +App::_AlertSimpleError(BMessage* message) +{ + BString alertTitle; + BString alertText; + + if (message->FindString(KEY_ALERT_TEXT, &alertText) != B_OK) + alertText = "?"; + + if (message->FindString(KEY_ALERT_TITLE, &alertTitle) != B_OK) + alertTitle = B_TRANSLATE("Error"); + + BAlert* alert = new BAlert(alertTitle, alertText, B_TRANSLATE("OK")); + + alert->SetFlags(alert->Flags() | B_CLOSE_ON_ESCAPE); + alert->Go(); +} + + // #pragma mark - private @@ -359,7 +386,7 @@ App::_LoadSettings(BMessage& settings) { if (!fSettingsRead) { fSettings = true; - if (load_settings(&fSettings, "main_settings", "HaikuDepot") != B_OK) + if (load_settings(&fSettings, KEY_MAIN_SETTINGS, "HaikuDepot") != B_OK) fSettings.MakeEmpty(); } settings = fSettings; @@ -390,7 +417,7 @@ App::_StoreSettings(const BMessage& settings) } } - save_settings(&fSettings, "main_settings", "HaikuDepot"); + save_settings(&fSettings, KEY_MAIN_SETTINGS, "HaikuDepot"); } diff --git a/src/apps/haikudepot/ui/App.h b/src/apps/haikudepot/ui/App.h index b596d73291..2fb02f8c08 100644 --- a/src/apps/haikudepot/ui/App.h +++ b/src/apps/haikudepot/ui/App.h @@ -1,5 +1,6 @@ /* * Copyright 2013, Stephan Aßmus . + * Copyright 2018, Andrew Lindesay * All rights reserved. Distributed under the terms of the MIT License. */ #ifndef APP_H @@ -25,6 +26,7 @@ public: virtual void ArgvReceived(int32 argc, char* argv[]); private: + void _AlertSimpleError(BMessage* message); void _Open(const BEntry& entry); void _ShowWindow(MainWindow* window); diff --git a/src/apps/haikudepot/ui/MainWindow.cpp b/src/apps/haikudepot/ui/MainWindow.cpp index 610eafb6d6..022011ab4b 100644 --- a/src/apps/haikudepot/ui/MainWindow.cpp +++ b/src/apps/haikudepot/ui/MainWindow.cpp @@ -32,30 +32,19 @@ #include #include -#include -#include -#include -#include -#include -#include "package/RepositoryCache.h" -#include -#include -#include -#include -#include - +#include "AppUtils.h" #include "AutoDeleter.h" #include "AutoLocker.h" #include "DecisionProvider.h" #include "FeaturedPackagesView.h" #include "FilterView.h" -#include "JobStateListener.h" #include "Logger.h" #include "PackageInfoView.h" #include "PackageListView.h" #include "PackageManager.h" +#include "ProcessCoordinator.h" +#include "ProcessCoordinatorFactory.h" #include "RatePackageWindow.h" -#include "RepositoryUrlUtils.h" #include "support.h" #include "ScreenshotWindow.h" #include "UserLoginWindow.h" @@ -67,7 +56,7 @@ enum { - MSG_MODEL_WORKER_DONE = 'mmwd', + MSG_BULK_LOAD_DONE = 'mmwd', MSG_REFRESH_REPOS = 'mrrp', MSG_MANAGE_REPOS = 'mmrp', MSG_SOFTWARE_UPDATER = 'mswu', @@ -75,6 +64,8 @@ enum { MSG_LOG_OUT = 'lgot', MSG_AUTHORIZATION_CHANGED = 'athc', MSG_PACKAGE_CHANGED = 'pchd', + MSG_WORK_STATUS_CHANGE = 'wsch', + MSG_WORK_STATUS_CLEAR = 'wscl', MSG_SHOW_FEATURED_PACKAGES = 'sofp', MSG_SHOW_AVAILABLE_PACKAGES = 'savl', @@ -133,10 +124,8 @@ MainWindow::MainWindow(const BMessage& settings) fLogInItem(NULL), fLogOutItem(NULL), fModelListener(new MessageModelListener(BMessenger(this)), true), - fBulkLoadStateMachine(&fModel), - fTerminating(false), - fSinglePackageMode(false), - fModelWorker(B_BAD_THREAD_ID) + fBulkLoadProcessCoordinator(NULL), + fSinglePackageMode(false) { BMenuBar* menuBar = new BMenuBar("Main Menu"); _BuildMenu(menuBar); @@ -216,7 +205,7 @@ MainWindow::MainWindow(const BMessage& settings) BPackageRoster().StartWatching(this, B_WATCH_PACKAGE_INSTALLATION_LOCATIONS); - _StartRefreshWorker(); + _StartBulkLoad(); _InitWorkerThreads(); } @@ -233,10 +222,8 @@ MainWindow::MainWindow(const BMessage& settings, const PackageInfoRef& package) fLogInItem(NULL), fLogOutItem(NULL), fModelListener(new MessageModelListener(BMessenger(this)), true), - fBulkLoadStateMachine(&fModel), - fTerminating(false), - fSinglePackageMode(true), - fModelWorker(B_BAD_THREAD_ID) + fBulkLoadProcessCoordinator(NULL), + fSinglePackageMode(true) { fFilterView = new FilterView(); fPackageListView = new PackageListView(fModel.Lock()); @@ -263,10 +250,6 @@ MainWindow::~MainWindow() { BPackageRoster().StopWatching(this); - fTerminating = true; - if (fModelWorker >= 0) - wait_for_thread(fModelWorker, NULL); - delete_sem(fPendingActionsSem); if (fPendingActionsWorker >= 0) wait_for_thread(fPendingActionsWorker, NULL); @@ -292,10 +275,12 @@ MainWindow::QuitRequested() StoreSettings(settings); BMessage message(MSG_MAIN_WINDOW_CLOSED); - message.AddMessage("window settings", &settings); + message.AddMessage(KEY_WINDOW_SETTINGS, &settings); be_app->PostMessage(&message); + _StopBulkLoad(); + return true; } @@ -304,14 +289,9 @@ void MainWindow::MessageReceived(BMessage* message) { switch (message->what) { - case MSG_MODEL_WORKER_DONE: - { - fModelWorker = B_BAD_THREAD_ID; - _AdoptModel(); - _UpdateAvailableRepositories(); - fWorkStatusView->SetIdle(); + case MSG_BULK_LOAD_DONE: + _BulkLoadCompleteReceived(); break; - } case B_SIMPLE_DATA: case B_REFS_RECEIVED: // TODO: ? @@ -320,11 +300,15 @@ MainWindow::MessageReceived(BMessage* message) case B_PACKAGE_UPDATE: // TODO: We should do a more selective update depending on the // "event", "location", and "change count" fields! - _StartRefreshWorker(false); + _StartBulkLoad(false); break; case MSG_REFRESH_REPOS: - _StartRefreshWorker(true); + _StartBulkLoad(true); + break; + + case MSG_WORK_STATUS_CHANGE: + _HandleWorkStatusChangeMessageReceived(message); break; case MSG_MANAGE_REPOS: @@ -685,8 +669,9 @@ void MainWindow::_BuildMenu(BMenuBar* menuBar) { BMenu* menu = new BMenu(B_TRANSLATE("Tools")); - menu->AddItem(new BMenuItem(B_TRANSLATE("Refresh repositories"), - new BMessage(MSG_REFRESH_REPOS))); + fRefreshRepositoriesItem = new BMenuItem( + B_TRANSLATE("Refresh repositories"), new BMessage(MSG_REFRESH_REPOS)); + menu->AddItem(fRefreshRepositoriesItem); menu->AddItem(new BMenuItem(B_TRANSLATE("Manage repositories" B_UTF8_ELLIPSIS), new BMessage(MSG_MANAGE_REPOS))); menu->AddItem(new BMenuItem(B_TRANSLATE("Check for updates" @@ -886,407 +871,123 @@ MainWindow::_ClearPackage() void -MainWindow::_RefreshRepositories(bool force) +MainWindow::_StopBulkLoad() { - if (fSinglePackageMode) - return; + AutoLocker lock(&fBulkLoadProcessCoordinatorLock); - BPackageRoster roster; - BStringList repositoryNames; - - status_t result = roster.GetRepositoryNames(repositoryNames); - if (result != B_OK) - return; - - DecisionProvider decisionProvider; - JobStateListener listener; - BContext context(decisionProvider, listener); - - BRepositoryCache cache; - for (int32 i = 0; i < repositoryNames.CountStrings(); ++i) { - const BString& repoName = repositoryNames.StringAt(i); - BRepositoryConfig repoConfig; - result = roster.GetRepositoryConfig(repoName, &repoConfig); - if (result != B_OK) { - // TODO: notify user - continue; - } - - if (roster.GetRepositoryCache(repoName, &cache) != B_OK || force) { - try { - BRefreshRepositoryRequest refreshRequest(context, repoConfig); - - result = refreshRequest.Process(); - } catch (BFatalErrorException ex) { - BString message(B_TRANSLATE("An error occurred while " - "refreshing the repository: %error% (%details%)")); - message.ReplaceFirst("%error%", ex.Message()); - message.ReplaceFirst("%details%", ex.Details()); - _NotifyUser("Error", message.String()); - } catch (BException ex) { - BString message(B_TRANSLATE("An error occurred while " - "refreshing the repository: %error%")); - message.ReplaceFirst("%error%", ex.Message()); - _NotifyUser("Error", message.String()); - } - } + if (fBulkLoadProcessCoordinator != NULL) { + printf("will stop full update process coordinator\n"); + fBulkLoadProcessCoordinator->Stop(); } } void -MainWindow::_RefreshPackageList(bool force) +MainWindow::_StartBulkLoad(bool force) { - if (fSinglePackageMode) - return; + AutoLocker lock(&fBulkLoadProcessCoordinatorLock); - if (Logger::IsDebugEnabled()) - printf("will refresh the package list\n"); - - BPackageRoster roster; - BStringList repositoryNames; - - status_t result = roster.GetRepositoryNames(repositoryNames); - if (result != B_OK) - return; - - std::vector depots(repositoryNames.CountStrings()); - for (int32 i = 0; i < repositoryNames.CountStrings(); i++) { - const BString& repoName = repositoryNames.StringAt(i); - DepotInfo depotInfo = DepotInfo(repoName); - - BRepositoryConfig repoConfig; - status_t getRepositoryConfigStatus = roster.GetRepositoryConfig( - repoName, &repoConfig); - - if (getRepositoryConfigStatus == B_OK) { - depotInfo.SetBaseURL(repoConfig.BaseURL()); - depotInfo.SetURL(repoConfig.URL()); - - if (Logger::IsDebugEnabled()) { - printf("local repository [%s] info;\n" - " * base url [%s]\n" - " * url [%s]\n", - repoName.String(), repoConfig.BaseURL().String(), - repoConfig.URL().String()); - } - } else { - printf("unable to obtain the repository config for local " - "repository '%s'; %s\n", - repoName.String(), strerror(getRepositoryConfigStatus)); - } - - depots[i] = depotInfo; + if (fBulkLoadProcessCoordinator == NULL) { + fBulkLoadProcessCoordinator + = ProcessCoordinatorFactory::CreateBulkLoadCoordinator( + this, + // PackageInfoListener + this, + // ProcessCoordinatorListener + &fModel, force); + fBulkLoadProcessCoordinator->Start(); + fRefreshRepositoriesItem->SetEnabled(false); } +} - PackageManager manager(B_PACKAGE_INSTALLATION_LOCATION_HOME); - try { - manager.Init(PackageManager::B_ADD_INSTALLED_REPOSITORIES - | PackageManager::B_ADD_REMOTE_REPOSITORIES); - } catch (BException ex) { - BString message(B_TRANSLATE("An error occurred while " - "initializing the package manager: %message%")); - message.ReplaceFirst("%message%", ex.Message()); - _NotifyUser("Error", message.String()); - return; - } - BObjectList packages; - result = manager.Solver()->FindPackages("", - BSolver::B_FIND_CASE_INSENSITIVE | BSolver::B_FIND_IN_NAME - | BSolver::B_FIND_IN_SUMMARY | BSolver::B_FIND_IN_DESCRIPTION - | BSolver::B_FIND_IN_PROVIDES, - packages); - if (result != B_OK) { - BString message(B_TRANSLATE("An error occurred while " - "obtaining the package list: %message%")); - message.ReplaceFirst("%message%", strerror(result)); - _NotifyUser("Error", message.String()); - return; - } +/*! This method is called when there is some change in the bulk load process. + A change may mean that a new process has started / stopped etc... or it + may mean that the entire coordinator has finished. +*/ - if (packages.IsEmpty()) - return; +void +MainWindow::CoordinatorChanged(ProcessCoordinatorState& coordinatorState) +{ + AutoLocker lock(&fBulkLoadProcessCoordinatorLock); - PackageInfoMap foundPackages; - // if a given package is installed locally, we will potentially - // get back multiple entries, one for each local installation - // location, and one for each remote repository the package - // is available in. The above map is used to ensure that in such - // cases we consolidate the information, rather than displaying - // duplicates - PackageInfoMap remotePackages; - // any package that we find in a remote repository goes in this map. - // this is later used to discern which packages came from a local - // installation only, as those must be handled a bit differently - // upon uninstallation, since we'd no longer be able to pull them - // down remotely. - BStringList systemFlaggedPackages; - // any packages flagged as a system package are added to this list. - // such packages cannot be uninstalled, nor can any of their deps. - PackageInfoMap systemInstalledPackages; - // any packages installed in system are added to this list. - // This is later used for dependency resolution of the actual - // system packages in order to compute the list of protected - // dependencies indicated above. - - for (int32 i = 0; i < packages.CountItems(); i++) { - BSolverPackage* package = packages.ItemAt(i); - const BPackageInfo& repoPackageInfo = package->Info(); - const BString repositoryName = package->Repository()->Name(); - PackageInfoRef modelInfo; - PackageInfoMap::iterator it = foundPackages.find( - repoPackageInfo.Name()); - if (it != foundPackages.end()) - modelInfo.SetTo(it->second); + if (fBulkLoadProcessCoordinator == coordinatorState.Coordinator()) { + if (!coordinatorState.IsRunning()) + _BulkLoadProcessCoordinatorFinished(coordinatorState); else { - // Add new package info - modelInfo.SetTo(new(std::nothrow) PackageInfo(repoPackageInfo), - true); - - if (modelInfo.Get() == NULL) - return; - - foundPackages[repoPackageInfo.Name()] = modelInfo; + _NotifyWorkStatusChange(coordinatorState.Message(), + coordinatorState.Progress()); + // show the progress to the user. } - - // The package list here considers those packages that are installed - // in the system as well as those that exist in remote repositories. - // It is better if the 'depot name' is from the remote repository - // because then it will be possible to perform a rating on it later. - - if (modelInfo->DepotName().IsEmpty() - || modelInfo->DepotName() == REPOSITORY_NAME_SYSTEM - || modelInfo->DepotName() == REPOSITORY_NAME_INSTALLED) { - modelInfo->SetDepotName(repositoryName); - } - - modelInfo->AddListener(this); - - BSolverRepository* repository = package->Repository(); - BPackageManager::RemoteRepository* remoteRepository = - dynamic_cast(repository); - - if (remoteRepository != NULL) { - - std::vector::iterator it; - - for (it = depots.begin(); it != depots.end(); it++) { - if (RepositoryUrlUtils::EqualsOnUrlOrBaseUrl( - it->URL(), remoteRepository->Config().URL(), - it->BaseURL(), remoteRepository->Config().BaseURL())) { - break; - } - } - - if (it == depots.end()) { - if (Logger::IsDebugEnabled()) { - printf("pkg [%s] repository [%s] not recognized" - " --> ignored\n", - modelInfo->Name().String(), repositoryName.String()); - } - } else { - it->AddPackage(modelInfo); - - if (Logger::IsTraceEnabled()) { - printf("pkg [%s] assigned to [%s]\n", - modelInfo->Name().String(), repositoryName.String()); - } - } - - remotePackages[modelInfo->Name()] = modelInfo; - } else { - if (repository == static_cast( - manager.SystemRepository())) { - modelInfo->AddInstallationLocation( - B_PACKAGE_INSTALLATION_LOCATION_SYSTEM); - if (!modelInfo->IsSystemPackage()) { - systemInstalledPackages[repoPackageInfo.FileName()] - = modelInfo; - } - } else if (repository == static_cast( - manager.HomeRepository())) { - modelInfo->AddInstallationLocation( - B_PACKAGE_INSTALLATION_LOCATION_HOME); - } - } - - if (modelInfo->IsSystemPackage()) - systemFlaggedPackages.Add(repoPackageInfo.FileName()); - } - - bool wasEmpty = fModel.Depots().IsEmpty(); - if (force || wasEmpty) - fBulkLoadStateMachine.Stop(); - - BAutolock lock(fModel.Lock()); - - if (force) - fModel.Clear(); - - // filter remote packages from the found list - // any packages remaining will be locally installed packages - // that weren't acquired from a repository - for (PackageInfoMap::iterator it = remotePackages.begin(); - it != remotePackages.end(); it++) { - foundPackages.erase(it->first); - } - - if (!foundPackages.empty()) { - BString repoName = B_TRANSLATE("Local"); - depots.push_back(DepotInfo(repoName)); - - for (PackageInfoMap::iterator it = foundPackages.begin(); - it != foundPackages.end(); ++it) { - depots.back().AddPackage(it->second); + } else { + if (Logger::IsInfoEnabled()) { + printf("unknown process coordinator changed\n"); } } - - { - std::vector::iterator it; - - for (it = depots.begin(); it != depots.end(); it++) { - if (fModel.HasDepot(it->Name())) - fModel.SyncDepot(*it); - else - fModel.AddDepot(*it); - } - } - - // start retrieving package icons and average ratings - if (force || wasEmpty) { - fBulkLoadStateMachine.Start(); - } - - // compute the OS package dependencies - try { - // create the solver - BSolver* solver; - status_t error = BSolver::Create(solver); - if (error != B_OK) - throw BFatalErrorException(error, "Failed to create solver."); - - ObjectDeleter solverDeleter(solver); - BPath systemPath; - error = find_directory(B_SYSTEM_PACKAGES_DIRECTORY, &systemPath); - if (error != B_OK) { - throw BFatalErrorException(error, - "Unable to retrieve system packages directory."); - } - - // add the "installed" repository with the given packages - BSolverRepository installedRepository; - { - BRepositoryBuilder installedRepositoryBuilder(installedRepository, - REPOSITORY_NAME_INSTALLED); - for (int32 i = 0; i < systemFlaggedPackages.CountStrings(); i++) { - BPath packagePath(systemPath); - packagePath.Append(systemFlaggedPackages.StringAt(i)); - installedRepositoryBuilder.AddPackage(packagePath.Path()); - } - installedRepositoryBuilder.AddToSolver(solver, true); - } - - // add system repository - BSolverRepository systemRepository; - { - BRepositoryBuilder systemRepositoryBuilder(systemRepository, - REPOSITORY_NAME_SYSTEM); - for (PackageInfoMap::iterator it = systemInstalledPackages.begin(); - it != systemInstalledPackages.end(); it++) { - BPath packagePath(systemPath); - packagePath.Append(it->first); - systemRepositoryBuilder.AddPackage(packagePath.Path()); - } - systemRepositoryBuilder.AddToSolver(solver, false); - } - - // solve - error = solver->VerifyInstallation(); - if (error != B_OK) { - throw BFatalErrorException(error, "Failed to compute packages to " - "install."); - } - - BSolverResult solverResult; - error = solver->GetResult(solverResult); - if (error != B_OK) { - throw BFatalErrorException(error, "Failed to retrieve system " - "package dependency list."); - } - - for (int32 i = 0; const BSolverResultElement* element - = solverResult.ElementAt(i); i++) { - BSolverPackage* package = element->Package(); - if (element->Type() == BSolverResultElement::B_TYPE_INSTALL) { - PackageInfoMap::iterator it = systemInstalledPackages.find( - package->Info().FileName()); - if (it != systemInstalledPackages.end()) - it->second->SetSystemDependency(true); - } - } - } catch (BFatalErrorException ex) { - printf("Fatal exception occurred while resolving system dependencies: " - "%s, details: %s\n", strerror(ex.Error()), ex.Details().String()); - } catch (BNothingToDoException) { - // do nothing - } catch (BException ex) { - printf("Exception occurred while resolving system dependencies: %s\n", - ex.Message().String()); - } catch (...) { - printf("Unknown exception occurred while resolving system " - "dependencies.\n"); - } - - if (Logger::IsDebugEnabled()) - printf("did refresh the package list\n"); } void -MainWindow::_StartRefreshWorker(bool force) +MainWindow::_BulkLoadProcessCoordinatorFinished( + ProcessCoordinatorState& coordinatorState) { - if (fModelWorker != B_BAD_THREAD_ID) - return; - - RefreshWorkerParameters* parameters = new(std::nothrow) - RefreshWorkerParameters(this, force); - if (parameters == NULL) - return; - - fWorkStatusView->SetBusy(B_TRANSLATE("Refreshing" B_UTF8_ELLIPSIS)); - - ObjectDeleter deleter(parameters); - fModelWorker = spawn_thread(&_RefreshModelThreadWorker, "model loader", - B_LOW_PRIORITY, parameters); - - if (fModelWorker > 0) { - deleter.Detach(); - resume_thread(fModelWorker); + if (coordinatorState.ErrorStatus() != B_OK) { + AppUtils::NotifySimpleError( + B_TRANSLATE("Package Update Error"), + B_TRANSLATE("While updating package data, a problem has arisen " + "that may cause data to be outdated or missing from the " + "application's display. Additional details regarding this " + "problem may be able to be obtained from the application " + "logs.")); } + BMessenger messenger(this); + messenger.SendMessage(MSG_BULK_LOAD_DONE); + // it is safe to delete the coordinator here because it is already known + // that all of the processes have completed and their threads will have + // exited safely by this point. + delete fBulkLoadProcessCoordinator; + fBulkLoadProcessCoordinator = NULL; + fRefreshRepositoriesItem->SetEnabled(true); } -status_t -MainWindow::_RefreshModelThreadWorker(void* arg) +void +MainWindow::_BulkLoadCompleteReceived() { - RefreshWorkerParameters* parameters - = reinterpret_cast(arg); - MainWindow* mainWindow = parameters->window; - ObjectDeleter deleter(parameters); + _AdoptModel(); + _UpdateAvailableRepositories(); + fWorkStatusView->SetIdle(); +} - BMessenger messenger(mainWindow); - mainWindow->_RefreshRepositories(parameters->forceRefresh); +/*! Sends off a message to the Window so that it can change the status view + on the front-end in the UI thread. +*/ - if (mainWindow->fTerminating) - return B_OK; +void +MainWindow::_NotifyWorkStatusChange(const BString& text, float progress) +{ + BMessage message(MSG_WORK_STATUS_CHANGE); - mainWindow->_RefreshPackageList(parameters->forceRefresh); + if (!text.IsEmpty()) + message.AddString(KEY_WORK_STATUS_TEXT, text); + message.AddFloat(KEY_WORK_STATUS_PROGRESS, progress); - messenger.SendMessage(MSG_MODEL_WORKER_DONE); + this->PostMessage(&message, this); +} - return B_OK; + +void +MainWindow::_HandleWorkStatusChangeMessageReceived(const BMessage* message) +{ + BString text; + float progress; + + if (message->FindString(KEY_WORK_STATUS_TEXT, &text) == B_OK) + fWorkStatusView->SetText(text); + + if (message->FindFloat(KEY_WORK_STATUS_PROGRESS, &progress) == B_OK) + fWorkStatusView->SetProgress(progress); } @@ -1455,17 +1156,6 @@ MainWindow::_PackagesToShowWorker(void* arg) } -void -MainWindow::_NotifyUser(const char* title, const char* message) -{ - BAlert* alert = new(std::nothrow) BAlert(title, message, - B_TRANSLATE("Close")); - - if (alert != NULL) - alert->Go(); -} - - void MainWindow::_OpenLoginWindow(const BMessage& onSuccessMessage) { diff --git a/src/apps/haikudepot/ui/MainWindow.h b/src/apps/haikudepot/ui/MainWindow.h index bb48ba83a6..9210824bcf 100644 --- a/src/apps/haikudepot/ui/MainWindow.h +++ b/src/apps/haikudepot/ui/MainWindow.h @@ -11,11 +11,11 @@ #include #include "TabView.h" -#include "BulkLoadStateMachine.h" #include "Model.h" #include "PackageAction.h" #include "PackageActionHandler.h" #include "PackageInfoListener.h" +#include "ProcessCoordinator.h" #include "HaikuDepotConstants.h" @@ -33,7 +33,7 @@ class WorkStatusView; class MainWindow : public BWindow, private PackageInfoListener, - private PackageActionHandler { + private PackageActionHandler, public ProcessCoordinatorListener { public: MainWindow(const BMessage& settings); MainWindow(const BMessage& settings, @@ -46,6 +46,9 @@ public: void StoreSettings(BMessage& message) const; + // ProcessCoordinatorListener + virtual void CoordinatorChanged( + ProcessCoordinatorState& coordinatorState); private: // PackageInfoListener virtual void PackageChanged( @@ -58,6 +61,9 @@ private: virtual Model* GetModel(); private: + void _BulkLoadProcessCoordinatorFinished( + ProcessCoordinatorState& + processCoordinatorState); bool _SelectedPackageHasWebAppRepositoryCode(); void _BuildMenu(BMenuBar* menuBar); @@ -73,19 +79,21 @@ private: void _AdoptPackage(const PackageInfoRef& package); void _ClearPackage(); - void _RefreshRepositories(bool force); - void _RefreshPackageList(bool force); - void _PopulatePackageAsync(bool forcePopulate); - void _StartRefreshWorker(bool force = false); + void _StopBulkLoad(); + void _StartBulkLoad(bool force = false); + void _BulkLoadCompleteReceived(); + + void _NotifyWorkStatusChange(const BString& text, + float progress); + void _HandleWorkStatusChangeMessageReceived( + const BMessage* message); + static status_t _RefreshModelThreadWorker(void* arg); static status_t _PackageActionWorker(void* arg); static status_t _PopulatePackageWorker(void* arg); static status_t _PackagesToShowWorker(void* arg); - void _NotifyUser(const char* title, - const char* message); - void _OpenLoginWindow( const BMessage& onSuccessMessage); void _UpdateAuthorization(); @@ -114,15 +122,15 @@ private: BMenuItem* fShowDevelopPackagesItem; BMenuItem* fShowSourcePackagesItem; + BMenuItem* fRefreshRepositoriesItem; + Model fModel; ModelListenerRef fModelListener; PackageList fVisiblePackages; - BulkLoadStateMachine - fBulkLoadStateMachine; + ProcessCoordinator* fBulkLoadProcessCoordinator; + BLocker fBulkLoadProcessCoordinatorLock; - bool fTerminating; bool fSinglePackageMode; - thread_id fModelWorker; thread_id fPendingActionsWorker; PackageActionList fPendingActions; diff --git a/src/apps/haikudepot/util/AppUtils.cpp b/src/apps/haikudepot/util/AppUtils.cpp new file mode 100644 index 0000000000..54933eff1c --- /dev/null +++ b/src/apps/haikudepot/util/AppUtils.cpp @@ -0,0 +1,32 @@ +/* + * Copyright 2018, Andrew Lindesay . + * All rights reserved. Distributed under the terms of the MIT License. + */ + + +#include "AppUtils.h" + +#include + +#include +#include + +#include "HaikuDepotConstants.h" + +/*! This method can be called to pop up an error in the user interface; + typically in a background thread. + */ + +/* static */ void +AppUtils::NotifySimpleError(const char* title, const char* text) +{ + BMessage message(MSG_ALERT_SIMPLE_ERROR); + + if (title != NULL && strlen(title) != 0) + message.AddString(KEY_ALERT_TITLE, title); + + if (text != NULL && strlen(text) != 0) + message.AddString(KEY_ALERT_TEXT, text); + + be_app->PostMessage(&message); +} \ No newline at end of file diff --git a/src/apps/haikudepot/util/AppUtils.h b/src/apps/haikudepot/util/AppUtils.h new file mode 100644 index 0000000000..1efae0abfe --- /dev/null +++ b/src/apps/haikudepot/util/AppUtils.h @@ -0,0 +1,18 @@ +/* + * Copyright 2018, Andrew Lindesay . + * All rights reserved. Distributed under the terms of the MIT License. + */ +#ifndef APP_UTILS_H +#define APP_UTILS_H + + +class AppUtils { + +public: + static void NotifySimpleError(const char* title, + const char* text); + +}; + + +#endif // APP_UTILS_H diff --git a/src/apps/haikudepot/util/DataIOUtils.cpp b/src/apps/haikudepot/util/DataIOUtils.cpp index 9f5f66cf2b..855ae363fe 100644 --- a/src/apps/haikudepot/util/DataIOUtils.cpp +++ b/src/apps/haikudepot/util/DataIOUtils.cpp @@ -2,8 +2,6 @@ * Copyright 2018, Andrew Lindesay . * All rights reserved. Distributed under the terms of the MIT License. */ - - #include "DataIOUtils.h" diff --git a/src/apps/haikudepot/util/RepositoryUrlUtils.cpp b/src/apps/haikudepot/util/RepositoryUrlUtils.cpp index 55365deacd..f12fb6e50d 100644 --- a/src/apps/haikudepot/util/RepositoryUrlUtils.cpp +++ b/src/apps/haikudepot/util/RepositoryUrlUtils.cpp @@ -32,19 +32,4 @@ RepositoryUrlUtils::EqualsNormalized(const BString& url1, const BString& url2) NormalizeUrl(normalizedUrl2); return normalizedUrl1 == normalizedUrl2; -} - - -/*! Matches on either the identifier URL of the repo or the 'base' URL that was - used to access the repository over the internet. The use of the 'base' URL - is deprecated. -*/ - -bool -RepositoryUrlUtils::EqualsOnUrlOrBaseUrl(const BString& url1, - const BString& url2, const BString& baseUrl1, - const BString& baseUrl2) -{ - return (!url1.IsEmpty() && url1 == url2) - || EqualsNormalized(baseUrl1, baseUrl2); } \ No newline at end of file diff --git a/src/apps/haikudepot/util/RepositoryUrlUtils.h b/src/apps/haikudepot/util/RepositoryUrlUtils.h index f7342a2e5f..3c05f48e3f 100644 --- a/src/apps/haikudepot/util/RepositoryUrlUtils.h +++ b/src/apps/haikudepot/util/RepositoryUrlUtils.h @@ -17,9 +17,6 @@ public: const BString& url2); static bool EqualsNormalized(const BUrl& normalizedUrl1, const BString& url2); - static bool EqualsOnUrlOrBaseUrl(const BString& url1, - const BString& url2, const BString& baseUrl1, - const BString& baseUrl2); }; diff --git a/src/apps/haikudepot/util/ToFileUrlProtocolListener.cpp b/src/apps/haikudepot/util/ToFileUrlProtocolListener.cpp index 6188d8dd35..2624f35cbb 100644 --- a/src/apps/haikudepot/util/ToFileUrlProtocolListener.cpp +++ b/src/apps/haikudepot/util/ToFileUrlProtocolListener.cpp @@ -1,5 +1,5 @@ /* - * Copyright 2017, Andrew Lindesay . + * Copyright 2017-2018, Andrew Lindesay . * All rights reserved. Distributed under the terms of the MIT License. */ @@ -59,7 +59,7 @@ ToFileUrlProtocolListener::HeadersReceived(BUrlRequest* caller, int32 statusCode = httpResult.StatusCode(); if (!BHttpRequest::IsSuccessStatusCode(statusCode)) { - fprintf(stdout, "received %" B_PRId32 + fprintf(stdout, "received http status %" B_PRId32 " --> will not store download to file\n", statusCode); fShouldDownload = false; } diff --git a/src/tests/apps/haikudepot/ListTest.cpp b/src/tests/apps/haikudepot/ListTest.cpp index 2b666e3cbe..aef64d10f4 100644 --- a/src/tests/apps/haikudepot/ListTest.cpp +++ b/src/tests/apps/haikudepot/ListTest.cpp @@ -1,5 +1,5 @@ /* - * Copyright 2017, Andrew Lindesay . + * Copyright 2017-2018, Andrew Lindesay . * All rights reserved. Distributed under the terms of the MIT License. */ @@ -15,6 +15,7 @@ #include "List.h" + ListTest::ListTest() { } @@ -38,23 +39,27 @@ static int32 CompareWithContextString(const void* context, const BString& str) } +/*! This tests the insertion of various letters into the list and the subsequent + search for those values later using a binary search. +*/ + void ListTest::TestBinarySearch() { - List list; + List list(&CompareStrings, &CompareWithContextString); BString tmp; for(char c = 'a'; c <= 'z'; c++) { tmp.SetToFormat("%c", c); - list.AddOrdered(tmp, &CompareStrings); + list.Add(tmp); } // ---------------------- - int32 aIndex = list.BinarySearch("a", &CompareWithContextString); - int32 hIndex = list.BinarySearch("h", &CompareWithContextString); - int32 uIndex = list.BinarySearch("u", &CompareWithContextString); - int32 zIndex = list.BinarySearch("z", &CompareWithContextString); - int32 ampersandIndex = list.BinarySearch("&", &CompareWithContextString); + int32 aIndex = list.Search("a"); + int32 hIndex = list.Search("h"); + int32 uIndex = list.Search("u"); + int32 zIndex = list.Search("z"); + int32 ampersandIndex = list.Search("&"); // ---------------------- CPPUNIT_ASSERT_EQUAL(0, aIndex); @@ -65,28 +70,34 @@ ListTest::TestBinarySearch() } +/*! In this test, a number of letters are added into a list. Later a check is + made to ensure that the letters were added in order. +*/ + void ListTest::TestAddOrdered() { - List list; + List list(&CompareStrings, NULL); // ---------------------- - list.AddOrdered(BString("p"), &CompareStrings); //1 - list.AddOrdered(BString("o"), &CompareStrings); - list.AddOrdered(BString("n"), &CompareStrings); - list.AddOrdered(BString("s"), &CompareStrings); - list.AddOrdered(BString("b"), &CompareStrings); //5 - list.AddOrdered(BString("y"), &CompareStrings); - list.AddOrdered(BString("r"), &CompareStrings); - list.AddOrdered(BString("d"), &CompareStrings); - list.AddOrdered(BString("i"), &CompareStrings); - list.AddOrdered(BString("k"), &CompareStrings); //10 - list.AddOrdered(BString("t"), &CompareStrings); - list.AddOrdered(BString("e"), &CompareStrings); - list.AddOrdered(BString("a"), &CompareStrings); - list.AddOrdered(BString("u"), &CompareStrings); - list.AddOrdered(BString("z"), &CompareStrings); // 15 - list.AddOrdered(BString("q"), &CompareStrings); + + list.Add(BString("p")); //1 + list.Add(BString("o")); + list.Add(BString("n")); + list.Add(BString("s")); + list.Add(BString("b")); //5 + list.Add(BString("y")); + list.Add(BString("r")); + list.Add(BString("d")); + list.Add(BString("i")); + list.Add(BString("k")); //10 + list.Add(BString("t")); + list.Add(BString("e")); + list.Add(BString("a")); + list.Add(BString("u")); + list.Add(BString("z")); // 15 + list.Add(BString("q")); + // ---------------------- CPPUNIT_ASSERT_EQUAL_MESSAGE("expected count of package infos", @@ -111,6 +122,52 @@ ListTest::TestAddOrdered() } +/*! This test will add a number of letters to a list which has no ordering. The + letters should then appear in the list in the order of insertion. +*/ + +void +ListTest::TestAddUnordered() +{ + List list; + +// ---------------------- + + list.Add(BString("b")); + list.Add(BString("e")); + list.Add(BString("t")); + list.Add(BString("h")); + list.Add(BString("e")); + list.Add(BString("l")); + list.Add(BString("l")); + list.Add(BString("s")); + list.Add(BString("b")); + list.Add(BString("e")); + list.Add(BString("a")); + list.Add(BString("c")); + list.Add(BString("h")); + +// ---------------------- + + CPPUNIT_ASSERT_EQUAL_MESSAGE("expected count of package infos", + 13, list.CountItems()); + + CPPUNIT_ASSERT_EQUAL(BString("b"), list.ItemAt(0)); + CPPUNIT_ASSERT_EQUAL(BString("e"), list.ItemAt(1)); + CPPUNIT_ASSERT_EQUAL(BString("t"), list.ItemAt(2)); + CPPUNIT_ASSERT_EQUAL(BString("h"), list.ItemAt(3)); + CPPUNIT_ASSERT_EQUAL(BString("e"), list.ItemAt(4)); + CPPUNIT_ASSERT_EQUAL(BString("l"), list.ItemAt(5)); + CPPUNIT_ASSERT_EQUAL(BString("l"), list.ItemAt(6)); + CPPUNIT_ASSERT_EQUAL(BString("s"), list.ItemAt(7)); + CPPUNIT_ASSERT_EQUAL(BString("b"), list.ItemAt(8)); + CPPUNIT_ASSERT_EQUAL(BString("e"), list.ItemAt(9)); + CPPUNIT_ASSERT_EQUAL(BString("a"), list.ItemAt(10)); + CPPUNIT_ASSERT_EQUAL(BString("c"), list.ItemAt(11)); + CPPUNIT_ASSERT_EQUAL(BString("h"), list.ItemAt(12)); +} + + /*static*/ void ListTest::AddTests(BTestSuite& parent) { @@ -127,5 +184,10 @@ ListTest::AddTests(BTestSuite& parent) "ListTest::TestBinarySearch", &ListTest::TestBinarySearch)); + suite.addTest( + new CppUnit::TestCaller( + "ListTest::TestAddUnordered", + &ListTest::TestAddUnordered)); + parent.addTest("ListTest", &suite); } \ No newline at end of file diff --git a/src/tests/apps/haikudepot/ListTest.h b/src/tests/apps/haikudepot/ListTest.h index 2b7d77fe3e..76d2dcdd52 100644 --- a/src/tests/apps/haikudepot/ListTest.h +++ b/src/tests/apps/haikudepot/ListTest.h @@ -1,5 +1,5 @@ /* - * Copyright 2017, Andrew Lindesay + * Copyright 2017-2018, Andrew Lindesay * Distributed under the terms of the MIT License. */ #ifndef LIST_TEST_H @@ -17,6 +17,7 @@ public: void TestAddOrdered(); void TestBinarySearch(); + void TestAddUnordered(); static void AddTests(BTestSuite& suite); };