11 December 2014 in The Review of Socionetwork Strategies
10 November 2020 in IEEE Transactions on Sustainable Computing
Popular data concentration is a widely accepted storage energy-saving technique which places frequently-accessed data on a small subset of hard disks and spins-down other infrequently-accessed disks. Many previous studies use intuitive heuristic algorithms for data placement that promote the imbalance in the access frequencies across hard disks. However, the relevance and the optimality of such file placements have not been rigorously investigated. In this paper, we formally define the energy-saving file placement problem under the capacity and performance constraints as a combinatorial optimization problem and show the theory of the optimal file placement where the file access rates in the next period are given. Our analysis based on a stochastic process of disk state transitions gives the theoretical support for the common heuristic placement method. To examine the effectiveness of the optimal file placement, we experimentally evaluate the energy-efficiency of a test storage system using the file access rates generated from the real access traces from Flickr. The experimental results show that the energy consumption can be reduced by 31.8% with the optimal file placement compared to the evenly distributed file placement. We also conduct simulation experiments to confirm the energy-saving impacts in larger-scale storage systems.
1 March 2016 in Future Generation Computer Systems
To empower scientists who are engaged in nation-wide or global-scale collaborative projects for scientific discovery, a large amount of scientific data needs to be visualized and then shared among the scientists. Tiled Display Wall (TDW) has been widely accepted and used for visualization of large-scale scientific data. Scalable Adaptive Graphics Environment (SAGE) has received attention from scientists as a middleware that organizes multiple display monitors into a network-aware large display monitor. Using a SAGE TDW, scientists can display multiple visualized contents on a single display monitor, each of which can be located at geographically distant site managed by other organizations. However, SAGE does not have a mechanism for managing multiple visualized data streams heading to a single TDW. In a conventional network, data flows for a same destination tend to share a same link, resulting in drop of packets and therefore poor visual quality. Moreover, because of the flexible nature of SAGE, rate of each visual data flow may change dynamically as a result of user interaction on a TDW, such as moving and resizing an application window. For the reason above, we propose and develop a dynamic route allocation method that switches packet flows onto network links where better performance is expected, in response to user interaction such as window movement and resizing. Technically, we have leveraged OpenFlow, an implementation of Software Defined Networking (SDN), to integrate network programmability into SAGE. In this paper, we show how SAGE enhanced with the proposed method succeeded in avoiding network congestion and improving the quality of visualization on the TDW over the wide area OpenFlow network on the Internet
13 November 2021 in IEEE Access
Big-data processing systems such as Hadoop, which usually utilize distributed file systems (DFSs), require data reduction schemes to maximize storage space efficiency. These schemes have different tradeoffs, and there are no all-purpose schemes applicable to all data. Users must select a suitable scheme in accordance with their data. To accommodate this requirement, application software or file system (FS) have a fixed selection of these schemes. However, these provided schemes are insufficient for all data types, and when novel schemes emerge, extending the selection can be problematic. If the source code of the application or FS is available, the source code could potentially be extended with extensive labor, but could be virtually impossible without the code maintainers’ assistance. If the source code is unavailable, there is no way to tackle the problem. This paper proposes an unexplored solution through a modular DFS design that eases data reduction scheme usage through existing programming techniques. The advantages of this presented approach are threefold. First, adding new schemes is easy and they are transparent to the application code requiring no extensions to it. Second, the modular structure requires minimal modification to the existing DFSs and performance overhead. Third, users can compile schemes separately from the DFS without the FS or DFS source code. To demonstrate the design’s effectiveness, we implemented it by minimally extending the Hadoop DFS (HDFS) and named it the Hadoop Data Reduction Framework (HDRF). We designed HDRF to work with minimal overhead and tested it extensively. Experimental results indicate that it has negligible overhead over existing approaches. In a number of cases, it can offer up to 48.96% higher throughput while achieving the best result in storage reduction within our tested setups because of the incorporated data reduction schemes.